Merge branch 'master' of github.com:phoronix-test-suite/phoronix-test-suite

https://github.com/phoronix-test-suite/phoronix-test-suite/issues/293

AUTHORS

@@ -1,8 +1,8 @@
-Copyright 2008 - 2014 by Phoronix Media.
+Copyright 2008 - 2018 by Phoronix Media.
 
 Lead Architects:
 
 - Michael Larabel <michael@phoronix.com>
 - Matthew Tippett <matthew@phoronix.com>
 
-There's also other individuals and organizations who have contributed patches, ideas, and made other contributions to the Phoronix Test Suite. These names are mentioned in the HTML / PDF documentation.
+There's also other individuals and organizations who have contributed patches, ideas, and made other contributions to the Phoronix Test Suite.

README.md

@@ -1,9 +1,8 @@
-Phoronix Test Suite 5.2.0m3
+# Phoronix Test Suite 8.4.0m2
 http://www.phoronix-test-suite.com/
-#####################################
 
-The Phoronix Test Suite is the most comprehensive testing and benchmarking
-platform available for Linux, Solaris, Mac OS X, and BSD operating systems. The
+The **Phoronix Test Suite** is the most comprehensive testing and benchmarking
+platform available for Linux, Solaris, OS X, and BSD operating systems. The
 Phoronix Test Suite allows for carrying out tests in a fully automated manner
 from test installation to execution and reporting. All tests are meant to be
 easily reproducible, easy-to-use, and support fully automated execution. The
@@ -28,18 +27,18 @@ able to upload their test results, profiles, and suites to OpenBenchmarking.org.
 A test profile is a single test that can be executed by the Phoronix Test Suite
 -- with a series of options possible within every test -- and a test suite is a
 seamless collection of test profiles and/or additional test suites. A test
-profile consists of a set of bash/shell scripts and XML files while a test suite
+profile consists of a set of Bash/shell scripts and XML files while a test suite
 is a single XML file.
 
-OpenBenchmarking.org <http://www.openbenchmarking.org/> also allows for
+[OpenBenchmarking.org](http://www.openbenchmarking.org/) also allows for
 conducting side-by-side result comparisons, a central location for storing and
-sharing test results, and collaborating over test data. Phoromatic
-<http://www.phoromatic.com/> is a complementary platform to OpenBenchmarking.org
-and the Phoronix Test Suite for interfacing with Phoronix Test Suite client(s)
-to automatically execute test runs on a timed, per-commit, or other
-trigger-driven basis. Phoromatic is designed for enterprise and allows for the
-easy management of multiple networked systems running Phoronix Test Suite
-clients via a single web-based interface.
+sharing test results, and collaborating over test data.
+[Phoromatic](http://www.phoromatic.com/) is a complementary platform to
+OpenBenchmarking.org and the Phoronix Test Suite for interfacing with Phoronix
+Test Suite client(s) to automatically execute test runs on a timed, per-commit,
+or other trigger-driven basis. Phoromatic is designed for enterprise and allows
+for the easy management of multiple networked systems running Phoronix Test
+Suite clients via a single web-based interface.
 
 Professional support and custom engineering for the Phoronix Test Suite,
 Phoromatic, and OpenBenchmarking.org is available by contacting
@@ -47,10 +46,9 @@ Phoromatic, and OpenBenchmarking.org is available by contacting
 
 Full details on the Phoronix Test Suite setup and usage is available from the
 included HTML/PDF documentation within the phoronix-test-suite package and from
-the Phoronix Test Suite web-site.
+the Phoronix Test Suite website.
 
-INSTALLATION & SETUP:
-#####################
+## Installation & Setup
 
 The Phoronix Test Suite is supported on Linux, *BSD, Solaris, Mac OS X, and
 Windows systems. However, the most full-featured and well supported operating
@@ -63,32 +61,31 @@ architectures.
 The Phoronix Test Suite can be installed for system-wide usage or run locally
 without installation from the extracted tar.gz/zip package. The only hard
 dependency on the Phoronix Test Suite is having command-line support for PHP
-(PHP 5.2+) installed. A complete PHP stack (e.g. with web-server) is NOT needed,
-but merely the PHP command-line support, which is widely available from
-operating system package managers under the name php, php5-cli, or php5.
+(PHP 5.3+) installed. A complete PHP stack (e.g. with web server) is **not**
+needed, but merely the PHP command-line support, which is widely available from
+operating system package managers under the name `php`, `php5-cli`, or `php5`.
 
-USAGE:
-######
+## Usage
 
 The process to download, install/setup, execute, and report the results of a
-benchmark can be as simple as a command such as *phoronix-test-suite benchmark
-smallpt* to run a simple CPU test profile. If wishing to simply install a test,
-it's a matter of running *phoronix-test-suite install <test or suite name>* and
-to run it's *phoronix-test-suite run <test or suite name>*. There's also a batch
-mode for non-interactive benchmarking by first running *phoronix-test-suite
-batch-setup* and then using the *batch-run* sub-command rather than *run*.
+benchmark can be as simple as a command such as `phoronix-test-suite benchmark
+smallpt` to run a simple CPU test profile. If wishing to simply install a test,
+it's a matter of running `phoronix-test-suite install <test or suite name>` and
+to run it's `phoronix-test-suite run <test or suite name>`. There's also a batch
+mode for non-interactive benchmarking by first running `phoronix-test-suite
+batch-setup` and then using the `batch-run` sub-command rather than `run`.
 
 Viewing installed system hardware and software is available via
-*phoronix-test-suite system-info* or *phoronix-test-suite detailed-system-info*
-for greater verbosity.
+`phoronix-test-suite system-info`.
 
 Facilitating a result comparison from OpenBenchmarking.org can be done by
-running, for example, *phoronix-test-suite benchmark 1204293-BY-PHORONIX357* if
+running, for example, `phoronix-test-suite benchmark 1204293-BY-PHORONIX357` if
 wishing to compare the results of the
-*http://openbenchmarking.org/result/1204293-BY-PHORONIX357* result file.
+`http://openbenchmarking.org/result/1204293-BY-PHORONIX357` result file.
 
-Additional information is available from the Phoronix Test Suite web-site
+Additional information is available from the Phoronix Test Suite website
 <http://www.phoronix-test-suite.com/> and the material bundled within the
-phoronix-test-suite/documentation/ directory. A man page is also bundled with
-the phoronix-test-suite software.
+`phoronix-test-suite/documentation/` directory or via
+<http://www.phoronix-test-suite.com/documentation/>. A man page is also bundled
+with the phoronix-test-suite software.
 

deploy/deb-package/build-package-deb.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2013, Phoronix Media
-	Copyright (C) 2008 - 2013, Michael Larabel
+	Copyright (C) 2008 - 2018, Phoronix Media
+	Copyright (C) 2008 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -23,9 +23,20 @@
 if(!is_executable("phoronix-test-suite") || !is_dir("pts-core/"))
 {
 	echo "\nYou must run this script from the root directory of the phoronix-test-suite/ folder!\n";
-	echo "Example: php5 pts-core/static/scripts/package-build-deb.php\n";
+	echo "Example: php deploy/deb-package/build-package-deb.php\n";
 	exit(0);
 }
+if(!is_executable('/usr/bin/dpkg'))
+{
+	echo PHP_EOL . "dpkg must be present on the system to generate the phoronix-test-suite Debian package." . PHP_EOL . PHP_EOL;
+	exit;
+}
+if(!is_executable('/usr/bin/fakeroot'))
+{
+	echo PHP_EOL . "fakeroot must be present on the system to generate the phoronix-test-suite Debian package." . PHP_EOL . PHP_EOL;
+	exit;
+}
+
 @require("pts-core/pts-core.php");
 
 if(!defined("PTS_VERSION"))
@@ -49,16 +60,17 @@ shell_exec("chmod +x /tmp/pts-deb-builder/usr/bin/phoronix-test-suite");
 $control_file = "Package: phoronix-test-suite\n";
 $control_file .= "Version: " . $pts_version . "\n";
 $control_file .= "Section: Utilities\n";
+$control_file .= "Installed-Size: " . shell_exec("cd /tmp/pts-deb-builder/; du -s | cut -f 1");
 $control_file .= "Priority: optional\n";
 $control_file .= "Architecture: all\n";
-$control_file .= "Depends: php5-cli, php5-gd, php5-json\n";
-$control_file .= "Recommends: build-essential\n";
+$control_file .= "Depends: php-cli|php5-cli,php5-cli|php-xml\n";
+$control_file .= "Recommends: build-essential, php-gd|php5-gd\n";
 $control_file .= "Maintainer: Phoronix Media <trondheim-pts@phoronix-test-suite.com>\n";
 $control_file .= "Description: An Automated, Open-Source Testing Framework\n " . @str_replace("\n", " ", file_get_contents('pts-core/static/short-description.txt')) . "\n";
 $control_file .= "Homepage: http://www.phoronix-test-suite.com/ \n";
 file_put_contents("/tmp/pts-deb-builder/DEBIAN/control", $control_file);
 
-shell_exec("dpkg --build /tmp/pts-deb-builder ../phoronix-test-suite_" . $pts_version . "_all.deb");
+shell_exec("fakeroot dpkg --build /tmp/pts-deb-builder ../phoronix-test-suite_" . $pts_version . "_all.deb");
 shell_exec("rm -rf /tmp/pts-deb-builder");
 
 ?>

deploy/deployments.md

@@ -0,0 +1,29 @@
+# Pre-Scripted Phoronix Test Suite / Phoromatic Deployments
+
+## deb-package: Generate A Debian/Ubuntu Package
+
+Running `php deploy/deb-package/build-package-deb.php` from the main `phoronix-test-suite/` directory will generate a basic Phoronix-Test-Suite Debian package. This script depends upon `fakeroot` and `dpkg` being present on the system.
+
+## rpm-package: Generate A RedHat/Fedora RPM Package
+
+Running `php deploy/rpm-package/build-package-rpm.php` from the main `phoronix-test-suite/` directory will generate a basic Phoronix-Test-Suite RPM package for Red Hat / Fedora based distributions. This script depends upon `rpmbuild` being present on the system.
+
+## phoromatic-upstart: Reference Upstart job files for Phoromatic
+
+The `*.conf` files provide basic `phoromatic-client` and `phoromatic-server` job files for Upstart-powered Linux systems looking to deploy the Phoromatic on either the front or back-end. Read the Phoronix Test Suite documentation for more details.
+
+## phoromatic-systemd: Reference systemd service files for Phoromatic
+
+The `*.service` files provide basic `phoromatic-client` and `phoromatic-server` job files for systemd-based Linux systems looking to deploy Phoromatic for controlling the Phoronix Test Suite.
+
+## phoromatic-initd: Reference sysvinit script for Phoromatic
+
+The files provide basic `phoromatic-client` /etc/init.d implementation for older Linux systems.
+
+## JuJu
+
+Ubuntu JuJu deployment charm for the Phoronix Test Suite.
+
+## farm-system-customizations: Example files of common system changes made to systems in the LinuxBenchmarking.com farm
+
+Various scripts commonly used by the Phoronix reference farm / LinuxBenchmarking.com for reference purposes.

deploy/farm-system-customizations/intel-xorg-headless.conf

@@ -0,0 +1,27 @@
+Section "Monitor"
+    Identifier             "Monitor0"
+    Modeline "1920x1080" 174.83 1920 2056 2248 2536 1080 1081 1084 1149             # 174.83 MHz,  68.94 kHz,  60.00 Hz
+    Option      "PreferredMode"  "1920x1080"
+        Option "Enable" "true"
+EndSection
+
+Section "Device"
+    Identifier             "Device0"
+    Driver                 "intel" #Choose the driver used for this monitor
+        Option "DDC" "0"
+        Option "Virtualheads" "1"
+        Option "Vsync" "0"
+        Option "SwapbuffersWait" "0"
+	Option "DRI" "3"
+EndSection
+
+Section "Screen"
+    Identifier             "Screen0"  #Collapse Monitor and Device section to Screen section
+    Device                 "Device0"
+    Monitor                "Monitor0"
+    DefaultDepth           24 #Choose the depth (16||24)
+    SubSection             "Display"
+        Depth              24
+        Modes              "1920x1080" #Choose the resolution
+    EndSubSection
+EndSection

deploy/farm-system-customizations/radeon-xorg-headless.conf

@@ -0,0 +1,23 @@
+Section "Device"
+    Identifier             "Device0"
+    Driver                 "radeon"
+	Option "SwapBuffersWait" "0"
+		Option "DRI "3"
+EndSection
+
+Section "Monitor"
+    Identifier             "Monitor0"
+        Option "enable" "true"
+     Modeline "1920x1080" 148.50 1920 2448 2492 2640 1080 1084 1089 1125 +hsync +vsync
+EndSection
+
+Section "Screen"
+        Identifier "Fake"
+        Device "Device0"
+        Monitor "Monitor0"
+        Option "ConnectedMonitor" "Monitor0"
+        SubSection "Display"
+                Depth 24
+                Modes "1920x1080"
+        EndSubSection
+EndSection

deploy/farm-system-customizations/ubuntu-initial-setup.sh

@@ -0,0 +1,10 @@
+#!/bin/sh
+
+sudo apt-get install openssh-server git-core php5-cli
+
+echo "GRUB_RECORDFAIL_TIMEOUT=0" >> /etc/default/grub
+update-grub
+
+sudo passwd
+
+# check if needing "tty -s &&" for auto log-in user in ~/.profile

deploy/phoromatic-initd/phoromatic-client

@@ -0,0 +1,73 @@
+#!/bin/sh
+#
+# /etc/init.d/phoromatic-client
+# Init script for Phoronix Test Suite Phoromatic client
+#
+# chkconfig: 2345 95 05
+# description: Starting the Phoronix Test Suite Phoromatic client
+#
+# processname: phoromatic-client
+# config: /etc/phoronix-test-suite.xml
+
+# source function library
+
+. /etc/rc.d/init.d/functions
+
+RETVAL=0
+prog="phoromatic-client"
+
+start() {
+	echo -n $"Starting $prog:"
+	env DISPLAY=:0
+	/usr/bin/phoronix-test-suite phoromatic.connect
+	RETVAL=$?
+	[ "$RETVAL" = 0 ] && touch /var/lock/subsys/$prog
+	echo
+}
+
+stop() {
+	echo -n $"Stopping $prog:"
+	killproc $prog -TERM
+	RETVAL=$?
+	[ "$RETVAL" = 0 ] && rm -f /var/lock/subsys/$prog
+	echo
+}
+
+reload() {
+	echo -n $"Reloading $prog:"
+	killproc $prog -HUP
+	RETVAL=$?
+	echo
+}
+
+case "$1" in
+	start)
+		start
+		;;
+	stop)
+		stop
+		;;
+	restart)
+		stop
+		start
+		;;
+	reload)
+		reload
+		;;
+	condrestart)
+		if [ -f /var/lock/subsys/$prog ] ; then
+			stop
+			sleep 6
+			start
+		fi
+		;;
+	status)
+		status $prog
+		RETVAL=$?
+		;;
+	*)
+		echo $"Usage: $0 {start|stop|restart|reload|condrestart|status}"
+		RETVAL=1
+esac
+exit $RETVAL
+

deploy/phoromatic-rcd/phoromatic-client

@@ -0,0 +1,6 @@
+#!/bin/sh
+# PROVIDE: phoromatic-client
+# REQUIRE: LOGIN
+
+/usr/bin/phoronix-test-suite phoromatic.connect
+

deploy/phoromatic-systemd/phoromatic-client.service

@@ -0,0 +1,14 @@
+[Unit]
+Description=Phoronix Test Suite Phoromatic Client
+After=syslog.target network-online.target
+Documentation=man:phoronix-test-suite(1)
+
+[Service]
+TimeoutStartSec=300
+ExecStart=/usr/bin/phoronix-test-suite phoromatic.connect
+Environment=DISPLAY=:0
+
+[Install]
+WantedBy=multi-user.target
+
+

deploy/phoromatic-systemd/phoromatic-server.service

@@ -0,0 +1,14 @@
+[Unit]
+Description=Phoronix Test Suite Phoromatic Server
+After=syslog.target network.target
+Documentation=man:phoronix-test-suite(1)
+
+[Service]
+TimeoutStartSec=300
+ExecStart=/usr/bin/phoronix-test-suite start-phoromatic-server
+Restart=on-failure
+
+[Install]
+WantedBy=multi-user.target
+
+

deploy/phoromatic-upstart/phoromatic-client.alt.conf

@@ -0,0 +1,10 @@
+# description     "Phoronix Test Suite Phoromatic Client"
+# author          "Phoronix Media"
+
+start on (local-filesystems and net-device-up IFACE!=lo)
+stop on stopping network
+stop on starting shutdown
+
+env DISPLAY=:0
+exec /usr/bin/phoronix-test-suite phoromatic.connect
+

deploy/phoromatic-upstart/phoromatic-client.conf

@@ -0,0 +1,8 @@
+description     "Phoronix Test Suite Phoromatic Client"
+author          "Phoronix Media"
+
+start on runlevel [35]
+stop on runlevel [0126]
+
+env DISPLAY=:0
+exec /usr/bin/phoronix-test-suite phoromatic.connect

deploy/phoromatic-upstart/phoromatic-server.conf

@@ -0,0 +1,10 @@
+# description     "Phoronix Test Suite Phoromatic Server"
+# author          "Phoronix Media"
+
+start on (local-filesystems and net-device-up IFACE!=lo)
+stop on stopping network
+stop on starting shutdown
+
+respawn
+
+exec /usr/bin/phoronix-test-suite start-phoromatic-server

deploy/rpm-package/build-package-rpm.php

@@ -3,7 +3,7 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2011, Phoronix Media
+	Copyright (C) 2008 - 2017, Phoronix Media
 	Copyright (C) 2008, Andrew Schofield
 
 	This program is free software; you can redistribute it and/or modify
@@ -20,12 +20,18 @@
 	along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 
-if(!is_file("phoronix-test-suite") || !is_dir("pts/") || !is_dir("pts-core/"))
+if(!is_file("phoronix-test-suite") || !is_dir("pts-core/"))
 {
        echo "\nYou must run this script from the root directory of the phoronix-test-suite/ folder!\n";
-       echo "Example: php5 pts-core/scripts/package-build-rpm.php\n";
+       echo "Example: php5 deploy/rpm-package/build-package-rpm.php\n";
        exit(0);
 }
+if(!is_executable('/usr/bin/rpmbuild'))
+{
+	echo PHP_EOL . "rpmbuild must be present on the system. Try: yum install rpm-build." . PHP_EOL . PHP_EOL;
+	exit;
+}
+
 @require("pts-core/pts-core.php");
 
 if(!defined("PTS_VERSION"))
@@ -34,21 +40,23 @@ if(!defined("PTS_VERSION"))
        exit(0);
 }
 
+$rpm_v = PTS_VERSION . '.' . date('YmdHi');
+
 shell_exec("rm -rf /tmp/pts-rpm-builder/");
-shell_exec("mkdir -p /tmp/pts-rpm-builder/{BUILD,RPMS,S{OURCE,PEC,RPM}S,phoronix-test-suite-" . PTS_VERSION . "}");
-shell_exec("cp -R ./ /tmp/pts-rpm-builder/phoronix-test-suite-" . PTS_VERSION . "/");
-shell_exec("tar --exclude=.git -C /tmp/pts-rpm-builder/ -cjvf /tmp/pts-rpm-builder/SOURCES/phoronix-test-suite-" . PTS_VERSION . ".tar.bz2 phoronix-test-suite-" . PTS_VERSION . "/");
+shell_exec("mkdir -p /tmp/pts-rpm-builder/{BUILD,RPMS,S{OURCE,PEC,RPM}S,phoronix-test-suite-" . $rpm_v . "}");
+shell_exec("cp -R ./ /tmp/pts-rpm-builder/phoronix-test-suite-" . $rpm_v . "/");
+shell_exec("tar --exclude=.git -C /tmp/pts-rpm-builder/ -cjvf /tmp/pts-rpm-builder/SOURCES/phoronix-test-suite-" . $rpm_v . ".tar.bz2 phoronix-test-suite-" . $rpm_v . "/");
 
 $spec_file = "Summary: A Comprehensive Linux Benchmarking System\n";
 $spec_file .= "Name: phoronix-test-suite\n";
-$spec_file .= "Version: " . PTS_VERSION . "\n";
+$spec_file .= "Version: " . $rpm_v . "\n";
 $spec_file .= "Release: 1\n";
 $spec_file .= "License: GPL\n";
 $spec_file .= "Group: Utilities\n";
 $spec_file .= "URL: http://www.phoronix-test-suite.com/\n";
-$spec_file .= "Source: phoronix-test-suite-" . PTS_VERSION . ".tar.bz2\n";
+$spec_file .= "Source: phoronix-test-suite-" . $rpm_v . ".tar.bz2\n";
 $spec_file .= "Packager: Phoronix Media <trondheim-pts@phoronix-test-suite.com>\n";
-$spec_file .= "Requires: php-cli, php-gd\n";
+$spec_file .= "Requires: php-cli, php-xml\n";
 $spec_file .= "BuildArch: noarch\n";
 $spec_file .= "BuildRoot: %{_tmppath}/%{name}-%{version}-root\n";
 $spec_file .= "%description\n";
@@ -65,18 +73,25 @@ $spec_file .= "rm -rf %{buildroot}\n";
 $spec_file .= "%files\n";
 $spec_file .= "%{_bindir}/phoronix-test-suite\n";
 $spec_file .= "%{_datadir}/phoronix-test-suite/*\n";
+$spec_file .= "%{_datadir}/applications/*\n";
+$spec_file .= "%{_datadir}/icons/hicolor/*\n";
+$spec_file .= "%{_datadir}/appdata/%{name}.appdata.xml\n";
 $spec_file .= "%{_datadir}/doc/*\n";
+$spec_file .= "%{_mandir}/man1/%{name}.1*\n";
+$spec_file .= "%config(noreplace) %{_sysconfdir}/bash_completion.d\n";
+$spec_file .= "%config(noreplace) %{_sysconfdir}/../usr/lib/systemd/system/*\n";
+//$spec_file .= "%config(noreplace) %{_sysconfdir}/init/*\n";
 $spec_file .= "%changelog\n";
-$spec_file .= "* Fri Jun 06 2008 Andrew Schofield <andrew_s@fahmon.net>\n";
+$spec_file .= "* " . date('D M d Y') . " Phoronix Media <phoronix@phoronix.com>\n";
 $spec_file .= "- Initial release.";
 
 file_put_contents("/tmp/pts-rpm-builder/SPECS/pts.spec", $spec_file);
-shell_exec("mv -f " . pts_client::user_home_directory() . ".rpmmacros /tmp/pts-rpm-builder");
-file_put_contents(pts_client::user_home_directory() .".rpmmacros", "%_topdir /tmp/pts-rpm-builder");
+shell_exec("mv -f " . getenv('HOME') . "/.rpmmacros /tmp/pts-rpm-builder 2>&1");
+file_put_contents(getenv('HOME') ."/.rpmmacros", "%_topdir /tmp/pts-rpm-builder");
 shell_exec("rpmbuild -ba --verbose /tmp/pts-rpm-builder/SPECS/pts.spec");
-shell_exec("cp /tmp/pts-rpm-builder/RPMS/noarch/phoronix-test-suite-" . PTS_VERSION . "-1.noarch.rpm ./");
-shell_exec("rm -f " . pts_client::user_home_directory() . "/.rpmmacros");
-shell_exec("mv -f /tmp/pts-rpm-builder/.rpmmacros " . pts_client::user_home_directory());
+shell_exec("cp /tmp/pts-rpm-builder/RPMS/noarch/phoronix-test-suite-" . $rpm_v . "-1.noarch.rpm ./");
+shell_exec("rm -f " . getenv('HOME') . "/.rpmmacros");
+shell_exec("mv -f /tmp/pts-rpm-builder/.rpmmacros " . getenv('HOME') . ' 2>&1');
 shell_exec("rm -rf /tmp/pts-rpm-builder");
 
 ?>

documentation/TODO

@@ -8,30 +8,26 @@ PHORONIX TEST SUITE PUBLIC TODO LIST:
 - Support for multiple languages / internationalization.
 - BLKID fall-back support for determining file-system [e.g. /sbin/blkid -s TYPE -o value /dev/sda1].
 - Possibly restore "weighted suites" [http://phorogit.com/index.php?p=phoronix-test-suite.git&a=commitdiff&h=548e5c073cc90d92ac3f382ca4933d5f556e78c6&hb=444441f9be866f51920e0ad8ca3909951b4d90c5]
-- Integrate recover-run command into the run command, auto-detect if the result file was a partial save and then ask if to perform run recovery' [ POSSIBLY DO THE SAME FOR FINISH-RUN]
+- Integrate finish-run command into the run command, auto-detect if the result file was a partial save and then ask if to perform run recovery'
 - Multi-threaded downloading of test files
-- Numerous pts_render improvements [see in-line code comments]
 - Optimus / DRI_PRIME=1 / xrandr --listproviders better GPU detection
 - NVIDIA make use of UsedDedicatedGPUMemory as new GPU sensor
 - Clean up all the WebSocket code and make it fit to spec
+- Incremental/streaming result uploads support
+- Check with Phodevi's parsers what other tools support JSON outputs - e.g. http://karelzak.blogspot.com/2015/06/json-output-for-basic-tools.html for cleaner parsing
+- Improve titles on tables like in: http://openbenchmarking.org/result/1603172-GA-AMDTONGA973
 
 ### PTS-GRAPH ###
 
-- Improve rendering of large data-sets, e.g. http://openbenchmarking.org/result/1205231-LI-BL460G6BE41
-- Rotated text alignment for some PNG graphixs is sometimes off by 90/180 degrees compared to SVG, e.g. http://openbenchmarking.org/s/Compiler
-- SVG graph improvements
-- WebP image support
+- Rotated text alignment for some PNG graphics is sometimes off by 90/180 degrees compared to SVG, e.g. http://openbenchmarking.org/s/Compiler
 
 ### OPENBENCHMARKING.ORG ###
 
 - Shorten long change-logs on test profile pages - e.g. http://openbenchmarking.org/test/pts/hpcc
-[ REST SUBJECT TO PHORONIX MEDIA NDA ]
 
-### NEW TEST PROFILES ###
+### POSSIBLE NEW TEST PROFILES ###
 
-- Python-libmemcached [http://code.google.com/p/python-libmemcached/]
-- Intel HiBench [https://github.com/intel-hadoop/HiBench]
-- Hadoop Benchmarks
+- Hadoop Benchmarks / TeraSort
 - LATT Latency Tests [git://git.kernel.dk/latt.git]
 - QMC Beaver [http://qmcbeaver.sourceforge.net/]
 - Some potential interesting GPGPU tests [http://www.bealto.com/gpu-benchmarks.html]
@@ -40,20 +36,44 @@ PHORONIX TEST SUITE PUBLIC TODO LIST:
 - SIPP [http://sipp.sourceforge.net/]
 - oclGaussCrack [https://hashcat.net/oclGaussCrack/]
 - http://hashcat.net/oclhashcat/
-- Handbrake [http://handbrake.fr/]
 - libMicro [https://github.com/portante/libMicro]
+- http://markjstock.org/pages/rad_bench.html
 - https://code.google.com/p/pyrit/
+- https://github.com/acangiano/ruby-benchmark-suite
 - http://www.nwchem-sw.org/index.php/Compiling_NWChem
 - http://sourceforge.net/projects/ffsb/
 - http://www.ratgpu.com/
 - http://panthema.net/2013/pmbw/
-- http://www.thedarkmod.com/
-- http://code.google.com/p/lz4/
-- http://parsec.cs.princeton.edu/parsec3-doc.htm
 - http://www.capsl.udel.edu/splash/Download.html
-- https://github.com/facebook/linkbench
-- https://github.com/vasi/pixz
-- http://nondot.org/sabre/LLVMNotes/#benchmarks
+- stephentu/silo
+- https://code.google.com/p/smhasher/wiki/SMHasher
+- http://benchmarksgame.alioth.debian.org/u32/performance.php?test=fasta / http://benchmarksgame.alioth.debian.org/u32/program.php?test=nbody&lang=gcc&id=1#sourcecode
+- https://openfoamwiki.net/index.php/Benchmarks_standard_v1
+- Cassandra 1.2 stress - http://docs.datastax.com/en/cassandra/1.2/cassandra/tools/toolsCStress_t.html
+- http://gcc.opensuse.org/c++bench-frescobaldi/
+- https://gooseberry.blender.org/gooseberry-production-benchmark-file/
+- https://github.com/gormanm/mmtests
+- https://github.com/soumith/convnet-benchmarks
+- http://apt.cs.manchester.ac.uk/projects/PAMELA/tools/SLAMBench/
+- https://haasn.xyz/posts/2017-10-05-how-to-benchmark-mpvs-raw-throughput.html
+- suitesparse http://faculty.cse.tamu.edu/davis/suitesparse.html
+- ospray/ospray
+- Microsoft/X-Mem
+- https://www2.mrc-lmb.cam.ac.uk/relion/index.php?title=Benchmarks_%26_computer_hardware
+- kripke
+- nekbone
+- https://github.com/u39kun/deep-learning-benchmark
+- AMG
+- CORAL-2 benchmarks
+- CompuBench (Windows)
+https://github.com/cart/amethyst-bunnymark
+rav1e (not yet multithreaded as of Oct 2018)
+gnzlbg/aobench
+https://github.com/mpicbg-scicomp/gearshifft
+https://github.com/hyperledger/caliper
+dav1d
+RedShift demo
+https://wiki.fysik.dtu.dk/gpaw/devel/benchmarks.html
 
 ### WEB SOCKET / HTML5 USER INTERFACE WORK ###
 
@@ -65,14 +85,7 @@ PHORONIX TEST SUITE PUBLIC TODO LIST:
 
 ### PHOROMATIC ####
 
-- Hookup the email notifications
-- Group handling
-- Get the Phoromatic search box implemented/working
-- Editing/deleting lots of different possible items like scheduling meta-data
 - Implement the complementary WebSocket communication to HTTP pulls
-- More mobile form factor optimizations
 - Ensure sane validating of more user input / resource uploads
-- Hook up the system targeting / groups support for test schedules
-- Restore some of the older phoromatic code for allowing reboots to happen during context script execution
 - Hook up TriggerTarget for allowing event.php to optionally specify a sub-set of systems/groups to target for a given trigger
 - Clean up the code and modularize more of the code into functions

documentation/man-pages/phoronix-test-suite.1

@@ -1,4 +1,4 @@
-.TH phoronix-test-suite 1  "www.phoronix-test-suite.com" "5.2.0m3"
+.TH phoronix-test-suite 1  "www.phoronix-test-suite.com" "8.4.0m2"
 .SH NAME
 phoronix-test-suite \- The Phoronix Test Suite is an extensible open-source platform for performing testing and performance evaluation.
 .SH SYNOPSIS
@@ -6,18 +6,40 @@ phoronix-test-suite \- The Phoronix Test Suite is an extensible open-source plat
 .br
 .B phoronix-test-suite benchmark [test | suite]
 .SH DESCRIPTION
-The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available for Linux, Solaris, Mac OS X, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners.
+The **Phoronix Test Suite** is the most comprehensive testing and benchmarking platform available for Linux, Solaris, OS X, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners.
 .SH OPTIONS
 .TP
-.SH TEST INSTALLATION
-.B install [Test | Suite | OpenBenchmarking.org ID | Test Result]  ...
-This option will install the selected test(s) inside the testing environment directory). The install process from downloading of the test files to the installation is fully automated. The install option needs to be supplied with the test name or suite as an argument. Optionally, a OpenBenchmarking.org ID or the name of a saved results file can be supplied as well and the test(s) to install will automatically be extracted from that information. If the test is already installed and was run by the latest version of the installation process, no action will be taken. Multiple arguments can be supplied to install additional tests at the same time.
+.SH SYSTEM
+.B diagnostics
+This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
 .TP
-.B install-dependencies [Test | Suite | OpenBenchmarking.org ID | Test Result]  ...
+.B interactive
+A simple text-driven interactive interface to the Phoronix Test Suite.
+.TP
+.B php-conf
+This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
+.TP
+.B shell
+A simple text-driven shell interface / helper to the Phoronix Test Suite. Ideal for those that may be new to the Phoronix Test Suite
+.TP
+.B system-info
+Display the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.
+.TP
+.B system-sensors
+Display the installed system hardware and software sensors in real-time as detected by the Phoronix Test Suite Phodevi Library.
+.TP
+.SH TEST INSTALLATION
+.B force-install [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem or wishing to re-install test(s) due to compiler or other environmental changes.
+.TP
+.B install [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option will install the selected test(s) inside the testing environment directory. The install process from downloading of the test files to the installation is fully automated. The install option needs to be supplied with the test name or suite as an argument. Optionally, a OpenBenchmarking.org ID or the name of a saved results file can be supplied as well and the test(s) to install will automatically be extracted from that information. If the test is already installed and was run by the latest version of the installation process, no action will be taken. Multiple arguments can be supplied to install additional tests at the same time.
+.TP
+.B install-dependencies [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will install the external dependencies needed by the selected test(s) using the distribution's package management system. For example, some tests depend upon GCC for compiling code. If GCC is not detected on the system, the Phoronix Test Suite will attempt to install GCC using the distribution's package management system. If you are running this command as a local user, you may be prompted for the root password while the process is running. For unsupported distributions, the dependency names will be displayed along with common names for the package. The install-dependencies option needs to be supplied with the test name or suite as an argument. When using the install option, the external dependencies are automatically checked.
 .TP
 .B make-download-cache
-This option will create a download cache for use by the Phoronix Test Suite.
+This option will create a download cache for use by the Phoronix Test Suite. The download cache is created of test files already downloaded to the local system. If passing any test/suite names to make-download-cache, the needed files for those test profiles will first be automatically downloaded before creating the cache.
 .TP
 .B remove-installed-test [Test]
 This option will permanently remove a installed test by the Phoronix Test Suite.
@@ -26,13 +48,16 @@ This option will permanently remove a installed test by the Phoronix Test Suite.
 .B auto-compare
 This option will autonomously determine the most relevant test(s) to run for any selected sub-system(s). The tests to run are determined via OpenBenchmarking.org integration with the global results pool. Related test results from OpenBenchmarking.org are also merged to provide a straight-forward and effective means of carrying out a system comparison. If wishing to find comparable results for any particular test profile(s), simply pass the test profile names as additional arguments to this command.
 .TP
-.B benchmark [Test | Suite | OpenBenchmarking.org ID | Test Result]  ...
+.B benchmark [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will install the selected test(s) (if needed) and will proceed to run the test(s). This option is equivalent to running phoronix-test-suite with the install option followed by the run option. Multiple arguments can be supplied to run additional tests at the same time and save the results into one file.
 .TP
+.B estimate-run-time [Test | Suite | OpenBenchmarking ID | Test Result]
+This option will provide estimates for test run-time / length.
+.TP
 .B finish-run [Test Result]
 This option can be used if a test run had not properly finished running all tests within a saved results file. Using this option when specifying a saved results file where all tests had not completed will attempt to finish testing on the remaining tests where there are missing results.
 .TP
-.B run [Test | Suite | OpenBenchmarking.org ID | Test Result]  ...
+.B run [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will run the selected test(s).
 .TP
 .B run-random-tests
@@ -41,41 +66,47 @@ This option will query OpenBenchmarking.org to run random benchmarks and result
 .B run-tests-in-suite
 This option can be used if you wish to run all of the tests found in a supplied suite, but you wish to re-configure each of the test options rather than using the defaults supplied by the suite.
 .TP
+.B stress-run [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option will run the passed tests/suites in the multi-process stress-testing mode. The stress-run mode will not produce a result file but is rather intended for running multiple test profiles concurrently to stress / burn-in the system. The number of tests to run concurrently can be toggled via the PTS_CONCURRENT_TEST_RUNS environment variable and by default is set to a value of 2.
+.TP
 .SH BATCH TESTING
-.B batch-benchmark [Test | Suite | OpenBenchmarking.org ID | Test Result]  ...
+.B batch-benchmark [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option and its arguments are equivalent to the benchmark option, but the process will be run in the Phoronix Test Suite batch mode.
 .TP
-.B batch-install [Test | Suite | OpenBenchmarking.org ID | Test Result]  ...
+.B batch-install [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 If you wish to run the install process in the Phoronix Test Suite batch mode but do not wish to run any tests at this time. Running the install process in the batch mode will use the default values and not prompt the user of any possible options, to ensure the process is fully automated.
 .TP
-.B batch-run [Test | Suite | OpenBenchmarking.org ID | Test Result]  ...
+.B batch-run [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option and its arguments are equivalent to the run option, but the process will be run in the Phoronix Test Suite batch mode.
 .TP
 .B batch-setup
 This option is used to configure the batch mode options for the Phoronix Test Suite, which is subsequently written to the user configuration file. Among the options are whether to automatically upload the test results to OpenBenchmarking.org and prompting for the saved file name.
 .TP
-.B default-benchmark [Test | Suite | OpenBenchmarking.org ID | Test Result]  ...
+.B default-benchmark [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will install the selected test(s) (if needed) and will proceed to run the test(s) in the defaults mode. This option is equivalent to running phoronix-test-suite with the install option followed by the default-run option.
 .TP
-.B default-run [Test | Suite | OpenBenchmarking.org ID | Test Result]  ...
+.B default-run [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will run the selected test(s). The name of the test or suite must be supplied or the OpenBenchmarking.org ID or saved local file name. Multiple arguments can be supplied to run additional tests at the same time and save the results in a suite-like fashion. Unlike the normal run option, the default-run will not prompt the user to select from the available test options but will instead use the default options as automatically set by pts-core or the test profile. Use batch-run to automatically test all of the available options.
 .TP
-.B internal-run [Test | Suite | OpenBenchmarking.org ID | Test Result]  ...
+.B dry-run [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option and its arguments pre-set the Phoronix Test Suite batch run mode with enforcing of defaults to not save any results and other behavior intended for a dry/test run. This option is primarily intended for testing/evaluation purposes.
+.TP
+.B internal-run [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option and its arguments pre-set the Phoronix Test Suite batch run mode with sane values for carrying out benchmarks in a semi-automated manner and without uploading any of the result data to the public OpenBenchmarking.org.
 .TP
 .SH OPENBENCHMARKING.ORG
 .B clone-result [OpenBenchmarking ID]  ...
-This option will download a local copy of a file that was saved to OpenBenchmarking.org, as long as a valid public ID is supplied. More than one ID can be specified and the results will then be merged.
+This option will download a local copy of a file that was saved to OpenBenchmarking.org, as long as a valid public ID is supplied.
 .TP
 .B list-recommended-tests
 This option will list recommended test profiles for benchmarking sorted by hardware sub-system. The recommended tests are determined via querying OpenBenchmarking.org and determining the most popular tests for a given environment based upon the number of times a test profile has been downloaded, the number of test results available on OpenBenchmarking.org for a given test profile, the age of the test profile, and other weighted factors.
 .TP
+.B make-openbenchmarking-cache
+This option will attempt to cache the test profile/suite meta-data from OpenBenchmarking.org for all linked repositories. This is useful if you're going to be running the Phoronix Test Suite / Phoromatic behind a firewall or without any Internet connection. Those with unrestricted Internet access or not utilizing a large local deployment of the Phoronix Test Suite / Phoromatic shouldn't need to run this command.
+.TP
 .B openbenchmarking-changes
 This option will list recent changes to test profiles of enabled OpenBenchmarking.org repositories.
 .TP
-.B openbenchmarking-launcher
-This option is called automatically with the .openbenchmarking MIME file extension support for launching OpenBenchmarking.org operations.
-.TP
 .B openbenchmarking-login
 This option is used for controlling your Phoronix Test Suite client options for OpenBechmarking.org and syncing the client to your account.
 .TP
@@ -85,6 +116,9 @@ This option is used for refreshing the stored OpenBenchmarking.org repostory inf
 .B openbenchmarking-repositories
 This option will list the OpenBenchmarking.org repositories currently linked to this Phoronix Test Suite client instance.
 .TP
+.B openbenchmarking-uploads
+This option will list any recent test result uploads from the system's IP address to OpenBenchmarking.org.
+.TP
 .B upload-result [Test Result]
 This option is used for uploading a test result to OpenBenchmarking.org.
 .TP
@@ -94,35 +128,25 @@ This option can be used for uploading a test profile to your account on OpenBenc
 .B upload-test-suite
 This option can be used for uploading a test suite to your account on OpenBenchmarking.org. By uploading your test suite to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution.
 .TP
-.SH SYSTEM
-.B detailed-system-info
-Display detailed information about the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.
-.TP
-.B diagnostics
-This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
-.TP
-.B interactive
-A simple text-driven interactive interface to the Phoronix Test Suite.
-.TP
-.B system-info
-Display the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.
-.TP
-.B system-sensors
-Display the installed system hardware and software sensors in real-time as detected by the Phoronix Test Suite Phodevi Library.
-.TP
 .SH INFORMATION
-.B info [Test | Suite | OpenBenchmarking.org ID | Test Result]
+.B info [Test | Suite | OpenBenchmarking ID | Test Result]
 This option will show details about the supplied test, suite, virtual suite, or result file.
 .TP
+.B list-all-tests
+This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories. Unlike the other test listing options, list-all-tests will show deprecated tests, potentially broken tests, or other tests not recommended for all environments. The only check in place is ensuring the test profiles are at least compatible with the operating system in use.
+.TP
 .B list-available-suites
 This option will list all test suites that are available from the enabled OpenBenchmarking.org repositories.
 .TP
 .B list-available-tests
-This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories.
+This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state. If the system has no Internet access, it will only list the test profiles where the necesary test assets are available locally on the system or on an available network cache (the same behavior as using the list-cached-tests sub-command), unless using the list-all-tests option to override this behavior.
 .TP
 .B list-available-virtual-suites
 This option will list all available virtual test suites that can be dynamically created based upon the available tests from enabled OpenBenchmarking.org repositories.
 .TP
+.B list-cached-tests
+This option will list all test profiles where any needed test profiles are already cached or available from the local system under test. This is primarily useful if testing offline/behind-the-firewall and other use-cases where wanting to rely only upon local data.
+.TP
 .B list-installed-dependencies
 This option will list all of the packages / external test dependencies that are already installed on the system that the Phoronix Test Suite may potentially depend upon by test profiles.
 .TP
@@ -135,6 +159,9 @@ This option will list all test profiles that are currently installed on the syst
 .B list-missing-dependencies
 This option will list all of the packages / external test dependencies that are missing from the system that the Phoronix Test Suite may potentially need by select test profiles.
 .TP
+.B list-not-installed-tests
+This option will list all test profiles that are supported and available but presently NOT installed on the system.
+.TP
 .B list-possible-dependencies
 This option will list all of the packages / external test dependencies that are are potentially used by the Phoronix Test Suite.
 .TP
@@ -147,21 +174,33 @@ This option will list various details about installed tests and their usage.
 .B list-unsupported-tests
 This option will list all available test profiles that are available from the enabled OpenBenchmarking.org repositories but are NOT SUPPORTED on the given hardware/software platform. This is mainly a debugging option for those looking for test profiles to potentially port to new platforms, etc.
 .TP
-.SH ASSET CREATION
-.B debug-install [Test | Suite | OpenBenchmarking.org ID | Test Result]  ...
-This option is intended for use by test profile writers and is identical to the install option but will yield more information during the run process that can be used to debug issues with a test profile installer or to verify the test profile is functioning correctly.
+.B search
+This option provides command-line searching abilities for test profiles / test suites / test results. The search query can be passed as a parameter otherwise the user is prompted to input their search query..
 .TP
-.B debug-run [Test | Suite | OpenBenchmarking.org ID | Test Result]  ...
+.SH ASSET CREATION
+.B build-suite
+This option will guide the user through the process of generating their own test suite, which they can then run. Optionally, passed as arguments can be the test(s) or suite(s) to add to the suite to be created, instead of being prompted through the process.
+.TP
+.B create-test-profile
+This option can be used for creating a Phoronix Test Suite test profile by answering questions about the test for constructing the test profile XML meta-data and handling other boiler-plate basics for getting started in developing new tests.
+.TP
+.B debug-benchmark [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option is intended for use by test profile writers and is identical to the <em>run</em> option but will yield more information during the run process that can be used to debug issues with a test profile or to verify the test profile is functioning correctly.
 .TP
-.B debug-test-download-links [Test | Suite]
+.B debug-install [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option is intended for use by test profile writers and is identical to the install option but will yield more information during the run process that can be used to debug issues with a test profile installer or to verify the test profile is functioning correctly.
+.TP
+.B debug-result-parser [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option is intended for use by test profile writers and is used for debugging a result parser. No test execution is done, but there must already be PTS-generated .log files present within the test's installation directory.
+.TP
+.B debug-test-download-links [Test | Suite | OpenBenchmarking ID | Test Result]
 This option will check all download links within the specified test profile(s) to ensure there are no broken URLs.
 .TP
-.B download-test-files [Test | Suite | OpenBenchmarking.org ID | Test Result]  ...
+.B download-test-files [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This will download the selected test file(s) to the Phoronix Test Suite download cache but will not install the tests.
 .TP
-.B force-install [Test | Suite | OpenBenchmarking.org ID | Test Result]  ...
-This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem.
+.B inspect-test-profile [Test]
+This option can be used for inspecting a Phoronix Test Suite test profile with providing inside details on test profiles for debugging / evaluation / learning purposes.
 .TP
 .B result-file-to-suite [Test Result]
 This option will guide the user through the process of generating their own test suite, which they can then run, that is based upon an existing test results file.
@@ -169,15 +208,18 @@ This option will guide the user through the process of generating their own test
 .B validate-result-file
 This option can be used for validating a Phoronix Test Suite result file as being compliant against the OpenBenchmarking.org specification.
 .TP
-.B validate-test-profile
+.B validate-test-profile [Test]
 This option can be used for validating a Phoronix Test Suite test profile as being compliant against the OpenBenchmarking.org specification.
 .TP
 .B validate-test-suite
 This option can be used for validating a Phoronix Test Suite test suite as being compliant against the OpenBenchmarking.org specification.
 .TP
 .SH RESULT MANAGEMENT
-.B copy-run-in-result-file [Test Result]
-This option is used if you wish to change an existing test run within a saved results file.
+.B analyze-all-runs [Test Result]
+This option will generate a candlestick graph showing the distribution of results from all trial runs. The candlestick graph is similar to the Japanese candlestick charts used by the financial industry, except instead of representing stock data it is numerical result data from all trial runs.\n\nThe tip of the upper-wick represents the highest value of the test runs with the tip of the lower-wick representing the lowest value of all test runs. The upper-edge of the candle body represents the first or last run value and the lower-edge represents the first or last run value. Lastly, if the last run value is less than the first run value, the candle body is the same color as the graph background, otherwise the last run value is greater.
+.TP
+.B auto-sort-result-file [Test Result]
+This option is used if you wish to automatically attempt to sort the results by their result identifier string.
 .TP
 .B edit-result-file [Test Result]
 This option is used if you wish to edit the title and description of an existing result file.
@@ -204,11 +246,14 @@ This option is used if you wish to change the name of the identifier in a test r
 This option is used if you wish to change the name of the saved name of a result file.
 .TP
 .B reorder-result-file [Test Result]
-This option is used if you wish to change the order in which test results are shown in the Phoronix Test Suite Results Viewer and the contained graphs. The user must specify a saved results file and then they will be prompted to select the results identifiers one at a time in the order they would like them to be displayed from left to right.
+This option is used if you wish to manually change the order in which test results are shown in the Phoronix Test Suite Results Viewer and the contained graphs. The user must specify a saved results file and then they will be prompted to select the results identifiers one at a time in the order they would like them to be displayed from left to right.
 .TP
 .B result-file-to-csv [Test Result]
 This option will read a saved test results file and output the system hardware and software information along with the results to a CSV output. The CSV (Comma Separated Values) output can then be loaded into a spreadsheet for easy viewing.
 .TP
+.B result-file-to-json [Test Result]
+This option will read a saved test results file and output the basic result information to JSON (JavaScript Object Notation).
+.TP
 .B result-file-to-pdf [Test Result]
 This option will read a saved test results file and output the system hardware and software information along with the results to a PDF file.
 .TP
@@ -218,52 +263,32 @@ This option will read a saved test results file and output the system hardware a
 .B show-result [Test Result]
 Open up the test results in the Phoronix Test Suite Result Viewer or on OpenBenchmarking.org.
 .TP
-.SH RESULT ANALYTICS
-.B analyze-all-runs [Test Result]
-This option will generate a candlestick graph showing the distribution of results from all trial runs. The candlestick graph is similar to the Japanese candlestick charts used by the financial industry, except instead of representing stock data it is numerical result data from all trial runs.\n\nThe tip of the upper-wick represents the highest value of the test runs with the tip of the lower-wick representing the lowest value of all test runs. The upper-edge of the candle body represents the first or last run value and the lower-edge represents the first or last run value. Lastly, if the last run value is less than the first run value, the candle body is the same color as the graph background, otherwise the last run value is greater.
-.TP
-.B analyze-batch [Test Result]
-This option will analyze a batch results file and plot out the performance impact from the different options onto a line graph (i.e. to see the impact that changing the video resolution has on the system's performance).
-.TP
-.B analyze-image-delta [Test Result]
-This option will analyze a test result file if it contains any test results that produced an image quality comparison (IQC) and will render image deltas illustrating the difference between images from two test results.
+.B winners-and-losers [Test Result]
+This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.
 .TP
 .SH OTHER
-.B build-suite
-This option will guide the user through the process of generating their own test suite, which they can then run. Optionally, passed as arguments can be the test(s) or suite(s) to add to the suite to be created, instead of being prompted through the process.
+.B commands
+This option will display a short list of possible Phoronix Test Suite commands.
+.TP
+.B debug-dependency-handler
+This option is used for testing the distribution-specific dependency handler for external dependencies.
+.TP
+.B debug-render-test
+This option is used during the development of the Phoronix Test Suite software for testing of the result and graph rendering code-paths This option will download a large number of reference test results from LinuxBenchmarking.com.
 .TP
 .B debug-self-test
 This option is used during the development of the Phoronix Test Suite software for testing of internal interfaces, commands, and other common code-paths. The produced numbers should only be comparable for the same version of the Phoronix Test Suite, on the same hardware/software system, conducted on the same day of testing. This isn't intended as any scientific benchmark but simply to stress common PHP code-paths and looking for hot areas to optimize, etc.
 .TP
 .B help
-This option will display the list of available Phoronix Test Suite commands.
-.TP
-.B network-setup
-This option allows the user to configure how the Phoronix Test Suite connects to OpenBenchmarking.org and other web-services. Connecting through an HTTP proxy can be configured through this option.
-.TP
-.B rebuild-composite-xml [Test Result]
-This option can be used to rebuild your composite XML file in the event that the main result file was corrupted and the test-X.xml files are still present in the result file directory.
-.TP
-.B user-config-reset
-This option can be used for resetting the Phoronix Test Suite user configuration file to its default state.
-.TP
-.B user-config-set
-This option can be used for setting an XML value in the Phoronix Test Suite user configuration file.
+This option will display a list of available Phoronix Test Suite commands and possible parameter types.
 .TP
 .B version
 This option will display the Phoronix Test Suite client version.
 .TP
-.SH WEB / GUI SUPPORT
-.B gui
-Launch the Phoronix Test Suite HTML5 web user-interface in the local GUI mode (no remote web support) and attempt to auto-launch the web-browser.
-.TP
-.B start-remote-gui-server
-Start the GUI web server and WebSocket server processes for remote (or local) access via the web-browser. The settings can be configured via the Phoronix Test Suite's user-config.xml configuration file.
-.TP
-.B start-ws-server
-Manually start a WebSocket server for communication by remote Phoronix Test Suite GUIs, the Phoronix Test Suite Multi-System Commander, and other functionality.
-.TP
 .SH MODULES
+.B auto-load-module
+This option can be used for easily adding a module to the LoadModules list in the Phoronix Test Suite user configuration file. That list controls what PTS modules are automatically loaded on start-up of the Phoronix Test Suite.
+.TP
 .B list-modules
 This option will list all of the available Phoronix Test Suite modules on this system.
 .TP
@@ -276,9 +301,32 @@ This option will allow you to configure all available end-user options for a Pho
 .B test-module [Phoronix Test Suite Module]
 This option can be used for debugging a Phoronix Test Suite module.
 .TP
-.SH GUI / WEB SUPPORT
-.B r
-p
+.B unload-module
+This option can be used for easily removing a module from the LoadModules list in the Phoronix Test Suite user configuration file. That list controls what modules are automatically loaded on start-up of the Phoronix Test Suite.
+.TP
+.SH USER CONFIGURATION
+.B enterprise-setup
+This option can be run by enterprise users immediately after package installation or as part of an in-house setup script. Running this command will ensure the phoronix-test-suite program is never interrupted on new runs to accept user agreement changes and defaults the anonymous usage reporting to being disabled and other conservative defaults.
+.TP
+.B network-info
+This option will print information detected by the Phoronix Test Suite around the system's network configuration.
+.TP
+.B network-setup
+This option allows the user to configure how the Phoronix Test Suite connects to OpenBenchmarking.org and other web-services. Connecting through an HTTP proxy can be configured through this option.
+.TP
+.B user-config-reset
+This option can be used for resetting the Phoronix Test Suite user configuration file to its default state.
+.TP
+.B user-config-set
+This option can be used for setting an XML value in the Phoronix Test Suite user configuration file.
+.TP
+.SH WEB / GUI SUPPORT
+.B gui
+Launch the Phoronix Test Suite HTML5 web user-interface in the local GUI mode (no remote web support) and attempt to auto-launch the web-browser. THIS FEATURE IS CURRENTLY EXPERIMENTAL AND NO LONGER ACTIVE DEVELOPMENT. See Phoronix Test Suite Phoromatic as an alternative web UI approach.
+.TP
+.SH PHOROMATIC
+.B n
+_
 .TP
 .SH SEE ALSO
 .B Websites:
@@ -293,5 +341,5 @@ http://www.phoronix.com/
 .br
 http://www.phoronix.com/forums/
 .SH AUTHORS
-Copyright 2008 - 2014 by Phoronix Media, Michael Larabel.
+Copyright 2008 - 2018 by Phoronix Media, Michael Larabel.
 .TP

documentation/old-docs/results-parser.xml

@@ -1,111 +0,0 @@
-<?xml version="1.0"?>
-<?xml-stylesheet type="text/xsl" href="xsl/pts-definitions-viewer.xsl" ?>
-<PhoronixTestSuite>
-	<Definitions>
-		<Define>
-			<Name>P_RESULTS_PARSER_MATCH_TO_TEST_ARGUMENTS</Name>
-			<Value>PhoronixTestSuite/ResultsParser/MatchToTestArguments</Value>
-			<Description>If there are multiple ResulsParser sections within the XML file and this tag is set, the Phoronix Test Suite will see if this string appears within the PTS_TEST_ARGUMENTS in determining if this is the ResultsParser section to use for parsing the results.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_TEMPLATE</Name>
-			<Value>PhoronixTestSuite/ResultsParser/OutputTemplate</Value>
-			<Description>The section of the test output to look for the result(s) within.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_RESULT_KEY</Name>
-			<Value>PhoronixTestSuite/ResultsParser/ResultKey</Value>
-			<Description>The key to look for when parsing the OutputTemplate to find where the result should be stored. If this tag is not implemented, the default key will be used.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_LINE_HINT</Name>
-			<Value>PhoronixTestSuite/ResultsParser/LineHint</Value>
-			<Description>A unique string to search the test result output for to indicate this is the line that contains the result value.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_LINE_BEFORE_HINT</Name>
-			<Value>PhoronixTestSuite/ResultsParser/LineBeforeHint</Value>
-			<Description>The LineBeforeHint tag is similar to LineHint but is used to indicate the line before where the test results appear, in the event the LineHint tag cannot be used reliably.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_LINE_AFTER_HINT</Name>
-			<Value>PhoronixTestSuite/ResultsParser/LineAfterHint</Value>
-			<Description>The LineAfterHint tag is similar to LineBeforeHint but is used to indicate the line after where the test results appear, in the event the LineHint or LineBeforeHint tags cannot be used reliably.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_RESULT_BEFORE_STRING</Name>
-			<Value>PhoronixTestSuite/ResultsParser/ResultBeforeString</Value>
-			<Description>The ResultBeforeString tag is optional and can be used if there is a consistient string following the reporting of the result, in the event the result's location is not static within the line string.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_STRIP_FROM_RESULT</Name>
-			<Value>PhoronixTestSuite/ResultsParser/StripFromResult</Value>
-			<Description>Any string that should be searched for and removed from the result value anywhere within the value string.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_STRIP_RESULT_POSTFIX</Name>
-			<Value>PhoronixTestSuite/ResultsParser/StripResultPostfix</Value>
-			<Description>This is similar to the StripFromResult tag but will only look at the end of the value string for this string to be removed. (I.e. if a unit such as 'ms' or 'FPS' is appended to the result by the test without a space.)</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_MULTI_MATCH</Name>
-			<Value>PhoronixTestSuite/ResultsParser/MultiMatch</Value>
-			<Description>If the result value should be found multiple times within the result file and then some mathematical operation carried out, it should be set using this tag. Setting the value to AVERAGE will find all matches and then calculate the average value based upon matches. While not required, setting the value to NONE will use the default code-path of only looking for one result value.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_DIVIDE_BY</Name>
-			<Value>PhoronixTestSuite/ResultsParser/DivideResultBy</Value>
-			<Description>If the result value should be divided by some constant (to convert the result to a different unit), the value should be supplied in this tag.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_MULTIPLY_BY</Name>
-			<Value>PhoronixTestSuite/ResultsParser/MultiplyResultBy</Value>
-			<Description>If the result value should be multiplied by some constant (to convert the result to a different unit), the value should be supplied in this tag.</Description>
-		</Define>
-		<Define>
-			<Name>P_IMAGE_PARSER_MATCH_TO_TEST_ARGUMENTS</Name>
-			<Value>PhoronixTestSuite/ImageParser/MatchToTestArguments</Value>
-			<Description>If there are multiple ResulsParser sections within the XML file and this tag is set, the Phoronix Test Suite will see if this string appears within the PTS_TEST_ARGUMENTS in determining if this is the ResultsParser section to use for parsing the results.</Description>
-		</Define>
-		<Define>
-			<Name>P_IMAGE_PARSER_SOURCE_IMAGE</Name>
-			<Value>PhoronixTestSuite/ImageParser/SourceImage</Value>
-			<Description>If the test profile is for an image quality comparison, the ImageFile tag should contain the file/path of the source file.</Description>
-		</Define>
-		<Define>
-			<Name>P_IMAGE_PARSER_IMAGE_X</Name>
-			<Value>PhoronixTestSuite/ImageParser/ImageX</Value>
-			<Description>If the test profile is for an image quality comparison, the ImageX should be the X coordinate of where to begin cropping the source file.</Description>
-		</Define>
-		<Define>
-			<Name>P_IMAGE_PARSER_IMAGE_Y</Name>
-			<Value>PhoronixTestSuite/ImageParser/ImageY</Value>
-			<Description>If the test profile is for an image quality comparison, the ImageX should be the Y coordinate of where to begin cropping the source file.</Description>
-		</Define>
-		<Define>
-			<Name>P_IMAGE_PARSER_IMAGE_WIDTH</Name>
-			<Value>PhoronixTestSuite/ImageParser/ImageWidth</Value>
-			<Description>If the test profile is for an image quality comparison, the ImageWidth should be the produced image width.</Description>
-		</Define>
-		<Define>
-			<Name>P_IMAGE_PARSER_IMAGE_HEIGHT</Name>
-			<Value>PhoronixTestSuite/ImageParser/ImageHeight</Value>
-			<Description>If the test profile is for an image quality comparison, the ImageHeight should be the produced image height.</Description>
-		</Define>
-		<Define>
-			<Name>P_MONITOR_PARSER_SENSOR</Name>
-			<Value>PhoronixTestSuite/SystemMonitor/Sensor</Value>
-			<Description>If the test profile is to monitor any system sensors during the testing process, the name of the sensor (via the Phodevi name; i.e. cpu.usage) should be supplied. While not a Phodevi sensor, if passing sys.time as the sensor value, the time it takes to execute the test will be recorded.</Description>
-		</Define>
-		<Define>
-			<Name>P_MONITOR_PARSER_FREQUENCY</Name>
-			<Value>PhoronixTestSuite/SystemMonitor/PollingFrequency</Value>
-			<Description>If the test profile is to monitor any system sensors during the testing process, how often the matched sensor is to be polled (in seconds) should be supplied. Floating values are supported.</Description>
-		</Define>
-		<Define>
-			<Name>P_MONITOR_PARSER_REPORT</Name>
-			<Value>PhoronixTestSuite/SystemMonitor/Report</Value>
-			<Description>If the test profile is to monitor any system sensors during the testing process, this tag provides how the monitored results should be reported. Supported values include MAX, MIN, AVG, and ALL.</Description>
-		</Define>
-	</Definitions>
-</PhoronixTestSuite>

documentation/old-docs/test-profile-downloads.xml

@@ -1,42 +0,0 @@
-<?xml version="1.0"?>
-<?xml-stylesheet type="text/xsl" href="xsl/pts-definitions-viewer.xsl" ?>
-<PhoronixTestSuite>
-	<Definitions>
-		<Define>
-			<Name>P_DOWNLOADS_PACKAGE_URL</Name>
-			<Value>PhoronixTestSuite/Downloads/Package/URL</Value>
-			<Description>The URL(s) to download the package from. If multiple URLs are supplied (delimited by a comma), the Phoronix Test Suite will randomly choose one URL but will fallback to other URLs if one fails either due to the file missing, the MD5 changed, or server inaccessability.</Description>
-			<Required>YES</Required>
-		</Define>
-		<Define>
-			<Name>P_DOWNLOADS_PACKAGE_MD5</Name>
-			<Value>PhoronixTestSuite/Downloads/Package/MD5</Value>
-			<Description>The MD5 check-sum for the file to be downloaded. This is to ensure file integrity. If no string is supplied, no MD5 check will be performed.</Description>
-			<Required>RECOMMENDED</Required>
-		</Define>
-		<Define>
-			<Name>P_DOWNLOADS_PACKAGE_FILENAME</Name>
-			<Value>PhoronixTestSuite/Downloads/Package/FileName</Value>
-			<Description>If this tag is set, once the file is downloaded it will be renamed to this value. If this tag is not set, the remote file-name will be used.</Description>
-			<Required>NO</Required>
-		</Define>
-		<Define>
-			<Name>P_DOWNLOADS_PACKAGE_FILESIZE</Name>
-			<Value>PhoronixTestSuite/Downloads/Package/FileSize</Value>
-			<Description>The size of the file to be downloaded should be the value of this tag. The FileSize should be an integer that is the number of bytes that makes up the file. This tag is used so that the Phoronix Test Suite can determine the total size of files that need to be downloaded for a given operation.</Description>
-			<Required>RECOMMENDED</Required>
-		</Define>
-		<Define>
-			<Name>P_DOWNLOADS_PACKAGE_PLATFORMSPECIFIC</Name>
-			<Value>PhoronixTestSuite/Downloads/Package/PlatformSpecific</Value>
-			<Description>If this package is just needed on select operating systems / platforms, list them inside this tag and multiple platforms can be delimited by a comma.</Description>
-			<Required>NO</Required>
-		</Define>
-		<Define>
-			<Name>P_DOWNLOADS_PACKAGE_ARCHSPECIFIC</Name>
-			<Value>PhoronixTestSuite/Downloads/Package/ArchitectureSpecific</Value>
-			<Description>If this package is just needed on select system/processor architectures, list them inside this tag and multiple architectures can be delimited by a comma.</Description>
-			<Required>NO</Required>
-		</Define>
-	</Definitions>
-</PhoronixTestSuite>

documentation/old-docs/writing_your_first_module.html

@@ -1,73 +0,0 @@
-<html xmlns="http://www.w3.org/1999/xhtml">
-<head>
-<Title>Phoronix Test Suite - Module Writing</Title>
-<meta http-equiv="content-type" content="text/html; charset=utf-8" />
-<link href="includes/pts-documentation.css" rel="stylesheet" type="text/css" />
-</head>
-<body>
-<div id="pts_doc_header"><div class="pts_doc_fixed"><a href="http://www.phoronix-test-suite.com/"><div id="pts_doc_logo"></div></a></div></div>
-<div class="pts_doc_fixed">
-<div class="pts_doc_notice"><div style="float: left"><a href="index.html">&lt;&lt; Documentation Home</a></div><div style="float: right;">Module Writing</div></div>
-<div class="pts_doc_main">
-<!-- PTS AREA -->
-<h1>Phoronix Test Suite Modules</h1>
-<div style="width: 1px; height: 20px;"></div>
-<p>Writing a module for the Phoronix Test Suite allows new functionality to be added 
-without having to extensively learn how pts-core functions. The module framework 
-for the Phoronix Test Suite allows modules to be written as a PHP object. Example PTS modules could include a module to shutdown the 
-screensaver when the Phoronix Test Suite starts up and re-enabling it when the 
-test is over, using sendmail to forward the results to an e-mail address when 
-testing is completed, or writing the current test status to a LCDproc-enabled 
-VFD display.</p>
-<p>Modules are stored in <em>pts-core/modules/</em>. Loading a 
-module is done by either setting the <em>PTS_MODULES</em> environmental variable 
-with the name of the module (excluding the <em>.php</em> file 
-extension) or by associating a module with a separate environmental variable. The default list of modules to be loaded is stored in <em>~/.phoronix-test-suite/user-config.xml</em>.</p>
-<p>Note: To run through all of the function calls for a module without needing to run a test, 
-run <em>phoronix-test-suite test-module MODULE_NAME</em>. Additionally, running 
-<em>phoronix-test-suite debug-module MODULE_NAME</em> will yield additional debugging details while 
-executing the same process.</p>
-<h1>Module</h1>
-<p>To see all of the functions supported for modules written in PHP, look at <em>pts-core/modules/dummy_module.php</em> 
-and additionally the other .php modules that ship with the Phoronix Test Suite. 
-Additionally, there are several functions written specifically for Phoronix Test 
-Suite modules that make it easier to save files, read files, and provided multi-threading 
-support for modules. The <em>pts_timed_function()</em> makes it possible (and 
-very easy) to thread functions within a module so that at a set interval the defined 
-functions will be called. For example, this support is used heavily within the 
-<em>system_monitor</em> module to poll sensors every X seconds even while there 
-are tests running. These functions can be found within <em>pts-core/objects/pts_module.php</em>.</p>
-<p>Below is a sample module that times how long it takes to run the Phoronix Test 
-Suite. It would be saved as <em>pts-core/modules/time_me.php</em>.</p>
-<blockquote>&lt;?php<br />
-class time_me extends pts_module_interface<br />
-{<br />
- &nbsp; &nbsp; const module_name = "Time Me!";<br />
- &nbsp; &nbsp; const module_version = "1.0.0";<br />
- &nbsp; &nbsp; const module_description = "This is a module that times how long the Phoronix Test Suite runs.";<br />
- &nbsp; &nbsp; const module_author = "Phoronix Media";<br />
-<br />
- &nbsp; &nbsp; static $start_time = NULL;<br />
- &nbsp; &nbsp; static $end_time = NULL;<br />
-<br />
- &nbsp; &nbsp; public static function __startup()<br />
- &nbsp; &nbsp; {<br />
- &nbsp; &nbsp;  &nbsp; &nbsp; self::$start_time = time();<br />
- &nbsp; &nbsp; }<br />
- &nbsp; &nbsp; public static function __shutdown()<br />
- &nbsp; &nbsp; {<br />
- &nbsp; &nbsp;  &nbsp; &nbsp; self::$end_time = time();<br />
-<br />
- &nbsp; &nbsp;  &nbsp; &nbsp; $time_elapsed = self::$end_time - self::$start_time;<br />
-<br />
- &nbsp; &nbsp;  &nbsp; &nbsp; echo "\nThe Phoronix Test Suite Ran For " . $time_elapsed . " Seconds.\n";<br />
- &nbsp; &nbsp; }<br />
-}<br />
-?&gt;</blockquote>
-<p>Then by running <em>PTS_MODULES=time_me phoronix-test-suite benchmark video-extensions</em>, 
-at the end of the test it would print a string similar to: &quot;The Phoronix 
-Test Suite Ran For 52 Seconds.&quot;</p>
-<!-- END OF PTS AREA -->
-</div></div><div class="pts_doc_fixed"><div class="pts_doc_bottom"><div style="float: left;"><a href="http://www.phoronix-test-suite.com/">Phoronix-Test-Suite.com</a></div><div style="float: right;">Copyright &copy; 2008 by <a href="http://www.phoronix-media.com/">Phoronix Media</a>.</div></div></div>
-</body>
-</html>

documentation/old-docs/writing_your_first_test.html

@@ -1,118 +0,0 @@
-<html xmlns="http://www.w3.org/1999/xhtml">
-<head>
-<Title>Phoronix Test Suite - A Quick Guide To Writing A Test Profile</Title>
-<link href="includes/pts-documentation.css" rel="stylesheet" type="text/css" />
-</head>
-<body>
-<div id="pts_doc_header"><div class="pts_doc_fixed"><a href="http://www.phoronix-test-suite.com/"><div id="pts_doc_logo"></div></a></div></div>
-<div class="pts_doc_fixed">
-<div class="pts_doc_notice"><div style="float: left"><a href="index.html">&lt;&lt; Documentation Home</a></div><div style="float: right;">Writing A Test Profile</div></div>
-<div class="pts_doc_main">
-<!-- PTS AREA -->
-<h1>Writing A Test Profile</h1>
-<p>Writing a test profile for the Phoronix Test Suite is a relatively quick and easy 
-process for anyone familiar with common Linux commands and the basics of XML. 
-To help you understand the design of the Phoronix Test Suite, this guide covers 
-the steps needed to write a testing profile for a very simple application.</p>
-<p>The first step in the profile writing process is to, well, have a piece of 
-software you'd like to use with the Phoronix Test Suite. This software can be 
-closed-source or open-source and be virtually anything as long as it is compatible 
-with the Linux operating system.</p>
-<p>For this guide, the piece of software being used for demonstration is just 
-a simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz 
-formula. Below is this sample piece of software intended just for demonstration 
-purposes.</p>
-<blockquote>#include &lt;iostream&gt;<br>
-#include &lt;math.h&gt;</p>
-<p>int main()<br>
-{<br>
-double pi = 0;</p>
-<p> for(long int i = 1; i &lt;= 87654321; i++)<br>
-pi += (double) pow(-1, i + 1) / (2 * i - 1);</p>
-<p> pi *= 4;<br>
-std::cout &lt;&lt; &quot;Done Calculating Pi...&quot; &lt;&lt; endl;<br>
-return 0;<br>
-}</blockquote>
-<p>The first step in the actual profile writing process is to name it. If you're 
-looking to ultimately push this profile to be included in the Phoronix Test Suite, 
-its name must be all lower case and consist of just alpha-numeric characters, 
-but can contain dashes (-). A more advanced test profile capability is operating system prefixes, and if using those there is an 
-underscore separating the prefix from the normal profile name. For this sample profile, we're calling it <em>sample-program</em> 
-and the file-name would be <em>sample-program.xml</em>. Our (very basic) profile is showcased 
-below.</p>
-<blockquote>&lt;PhoronixTestSuite&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestProfile&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Version&gt;1.1.0&lt;/Version&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestType&gt;Processor&lt;/TestType&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;SoftwareType&gt;Utility&lt;/SoftwareType&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;License&gt;FREE&lt;/License&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Status&gt;PRIVATE&lt;/Status&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Maintainer&gt;Phoronix Media&lt;/Maintainer&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/TestProfile&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestInformation&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Title&gt;Sample Pi Program&lt;/Title&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TimesToRun&gt;3&lt;/TimesToRun&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ResultScale&gt;Seconds&lt;/ResultScale&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Proportion&gt;LIB&lt;/Proportion&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Description&gt;A simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz formula. This test can be used for showcasing how to write a basic test profile.&lt;/Description&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ExternalDependencies&gt;build-utilities&lt;/ExternalDependencies&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/TestInformation&gt;<br>
-&lt;/PhoronixTestSuite&gt;</blockquote>
-<p>This XML profile is what interfaces with the Phoronix Test Suite and provides 
-all the needed information about the test as well as other attributes. For a complete 
-listing of all the supported profile options, look at the specification files 
-in the documentation folder. In the case of <em>sample-program</em>, it lets the Phoronix 
-Test Suite know that it's composed of free software, is designed to test the processor, 
-is intended for private use only, and this profile is maintained by Phoronix Media. 
-In addition, it tells the Phoronix Test Suite to execute this program three times 
-and as no result quantifier is set, the average of the three runs will be taken. 
-This profile also tells the Phoronix Test Suite that the generic <em>build-utilities</em> 
-package is needed, which will attempt to ensure that GCC and the standard development 
-utilities/libraries are installed on your Linux distribution. This is needed as 
-the C++ source-code will need to be built from source.</p>
-<p>The next step is to write the <em>install.sh</em> file, which once called by the Phoronix 
-Test Suite is intended to install the test locally for benchmarking purposes. 
-The <em>install.sh</em> file is technically optional, but is generally used by all tests. Note: The first argument supplied to 
-the install script is the directory that the test needs to be installed to. The 
-<em>install.sh</em> file (in our instance) is to be placed inside <em>pts/test-resources/sample-program</em>. 
-Below is the <em>install.sh</em> for the <em>sample-program</em>.</p>
-<blockquote>#!/bin/sh<br /><br />
-tar -xjf sample-pi-program-1.tar.bz2<br />
-g++ sample-pi-program.cpp -o sample-pi-program<br />
-echo &quot;#!/bin/sh<br />
-\$TIMER_START<br />
-./sample-pi-program 2&gt;&amp;1<br />
-\$TIMER_STOP<br />
-&quot; &gt; sample-program<br>
-chmod +x sample-program</blockquote>
-<p>Phoronix Test Suite 1.4 and later provides an integrated multi-platform micro-timer framework that provides the <em>$TIMER_START</em> and <em>$TIMER_STOP</em> functionality.</p>
-<p>This install file builds the code with GCC, and then creates a small script 
-that is run by the Phoronix Test Suite, which times how long it takes to run the 
-software. Where does the source-code come into play? Well, it needs to be downloaded 
-now from a web server. The Phoronix Test Suite has built-in support for managing 
-downloads from multiple servers in a random over, fall-back support if one mirror 
-is done, and verification of MD5 check-sums. Below is the <em>downloads.xml</em> file for 
-<em>sample-program</em> that covers all of this.</p>
-<blockquote>&lt;PhoronixTestSuite&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Downloads&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Package&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;URL&gt;http://www.phoronix-test-suite.com/benchmark-files/sample-pi-program.cpp&lt;/URL&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;MD5&gt;e90fb790df8d1544696a1439c9b5bd8d&lt;/MD5&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/Package&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/Downloads&gt;<br>
-&lt;/PhoronixTestSuite&gt;</blockquote>
-<p>The final step in the profile writing process is to write a parser to strip 
-all information but the reported result from the standard output. The standard 
-output is submitted to <em>parse-results.sh</em> or <em>parse-results.php</em> as the first argument 
-in quotes or using <em>$LOG_FILE</em> if the test profile writes to that variable's location.</p>
-<p>If the test profile uses the integrated micro-timer framework with <em>$TIMER_START</em> and <em>$TIMER_STOP</em> but no parse-results file is provided, the Phoronix Test Suite will automatically use the difference between <em>$TIMER_START</em> and <em>$TIMER_STOP</em> and use that as the test result. This is the time (in seconds) that elapsed between starting and stopping the timer.</p>
-<p>After that, with all the files in their correct locations, just run: <em>phoronix-test-suite 
-benchmark sample-program</em>. The Phoronix Test Suite should now handle the rest by 
-installing the test, running the test, and recording the results (if you so choose). 
-There is no additional work that needs to be done for the results to be recorded 
-in the PTS Results Viewer or even reporting the results to OpenBenchmarking.org.</p>
-<p><strong>For more information, visit <a href="http://www.phoronix-test-suite.com/">Phoronix-Test-Suite.com</a> and read the <a href="index.html">included documentation</a>. It should also be of great help to just look at the existing test profiles, which can be found inside the <em>phoronix-test-suite/pts/</em> folder with the <em>test-resources</em> and <em>test-profiles</em> sub-directories.</strong></p>
-<!-- END OF PTS AREA -->
-</div></div><div class="pts_doc_fixed"><div class="pts_doc_bottom"><div style="float: left;"><a href="http://www.phoronix-test-suite.com/">Phoronix-Test-Suite.com</a></div><div style="float: right;">Copyright &copy; 2008 by <a href="http://www.phoronix-media.com/">Phoronix Media</a>.</div></div></div>
-</body>
-</html>

documentation/phoromatic.html

@@ -0,0 +1,90 @@
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+<Title>Phoronix Test Suite Phoromatic</Title>
+</head>
+<body>
+<h1>Phoromatic Server</h1>
+<h2>Introduction</h2>
+<p>Phoromatic is a remote management system for the Phoronix Test Suite. Phoromatic allows the automatic (hence the name <em>Phoro-matic</em>) scheduling of tests, remote installation of new tests, and the management of multiple test systems all through an intuitive, easy-to-use web interface. Tests can be scheduled to automatically run on a routine basis across multiple test systems. The test results are then available from this central, secure location.</p>
+<p>Phoromatic was originally introduced with Phoronix Test Suite 2.0 via Phoromatic.com as a project going back to 2008~2009. Phoromatic.com debuted as a hosted instance with the option of behind-the-firewall licensing for use within organizations. With Phoronix Test Suite 5.2 the model shifted to offer a local, open-source version of Phoromatic built into the Phoronix Test Suite code-base. Thanks to continued enterprise development, with Phoronix Test Suite 5.4 is now a fully-functioning, built-in version of Phoromatic that's open-source and can be used for behind-the-firewall testing without needing to push results to OpenBenchmarking.org and the ability to keep all results private.</p>
+<p>Phoromatic in Phoronix Test Suite 5.4 also has the ability to support zero-conf  network discovery using Avahi and the automatic distribution of needed test profiles/suites and test files. Phoronix Test Suite 5.4's Phoromatic is a significant breakthrough for open-source testing particularly those running this GPL benchmarking software within test labs and other large organizations.</p>
+<h2>Features</h2>
+<p>Built atop the Phoronix Test Suite, Phoromatic offers many features for both enterprise and community/personal users:</p>
+<h3>Automated Scheduling</h3>
+<p>Whether it is every evening at 6:00PM, once every Thursday at 10:00AM or somewhere in between, Phoromatic can schedule tests to be run at user-defined intervals. The testing schedules can be updated through Phoromatic web interface. After the test(s) have run, the results will be immediately uploaded to Phoromatic.</p>
+<h3>Extensible</h3>
+<p>Any test profile or test suite that is compliant with the Phoronix Test Suite specification will work with Phoromatic. Phoromatic is able to leverage the hundreds of test profiles and test suites currently in the Phoronix Test Suite via OpenBenchmarking.org, along with any custom or proprietary test profiles you or your company utilize. Additionally, the Phoromatic interface allows the user to construct their own test suite(s).</p>
+<h3>Remote Testing</h3>
+<p>Once the test system is setup, all testing and management of that system can be done remotely. There is no need to execute Phoronix Test Suite commands locally using the GUI or command line version, but instead nearly all of the same features are accessible from the Phoromatic interface.</p>
+<h3>Multi-System Support</h3>
+<p>A single Phoromatic account is able to manage multiple test systems running the Phoronix Test Suite. Phoromatic supports grouping together test systems, tagging, and other features to support effectively managing many test systems. From the Phoromatic interface, installed system hardware and software from a given system can also be viewed.</p>
+<h3>Turn-Key Deployment</h3>
+<p>No additional software needs to be installed to support Phoromatic; all that's needed is Phoronix Test Suite 5.4 or later for full compatibility. New test systems can easily be synced with a given Phoromatic account by running a single command from the Phoronix Test Suite client.</p>
+<h3>Result Management</h3>
+<p>Test results are automatically uploaded to the Phoromatic account and remain private unless you opt to upload them to OpenBenchmarking.org. From the Phoromatic interface, results from multiple test systems can easily be compared and multiple results from the same systems can be used to track performance over time. There are also options to look at the statistical significance of the results and other features to efficiently and effectively analyze the system's performance.</p>
+<h3>Decentralized</h3>
+<p>Once the Phoronix Test Suite running on the Phoromatic Server has been able to cache all of the OpenBenchmarking.org test files and the needed files for each test, Phoromatic with any Phoronix Test Suite clients on your LAN can run fully decentralized without the need for a constant stream of OpenBenchmarking.org communication or Internet connection for that matter. (The only exception would be if your local systems don't have all their needed external dependencies and your system's package manager would need to install components like a compiler or necessary system libraries.</p>
+<h3>Fully Open-Source</h3>
+<p>Phoromatic is now fully open-source within the Phoronix Test Suite code-base for fostering greater development and new capabilities. Patches are welcome and Phoronix Media is available to provide commercial support and custom engineering services around Phoromatic and the Phoronix Test Suite.</p>
+<h2>Phoromatic Server Setup</h2>
+<p>Phoromatic is built into the Phoronix Test Suite code-base and should be found in all packaged versions of the <strong>phoronix-test-suite</strong>. Starting the Phoromatic Server entails running phoronix-test-suite start-phoromatic-server after configuring the server information within <em>~/.phoronix-test-suite/user-config.xml</em>. The Phoromatic Server can with or without root permissions depending upon your firewall and the port numbers you wish to use for the server.</p>
+<p>On the "client side", any up-to-date version of the Phoronix Test Suite can automatically communicate with the Phoromatic Server. If Avahi support is available (commonly in Linux distribution repositories as <u>avahi-tools</u>), there should be zero-conf discovery if the Phoromatic Server and client systems are on the same LAN. If a Phoronix Test Suite client discovers a Phoromatic Server, it will attempt to use it automatically as a local download cache. In the event of no Internet connection, it will also attempt to obtain the needed OpenBenchmarking.org test/suite meta-data from the Phoromatic Server based upon its archived meta-data. This allows the Phoronix Test Suite / Phoromatic deployment on the LAN to be self-sustaining without an Internet connection as long as the systems have all installed test dependencies.</p>
+<p>Further configuration of the setup parameters for the Phoromatic Server and Phoronix Test Suite clients can be tuned via the <em>~/.phoronix-test-suite/user-config.xml</em> file. All control and configuration of the Phoromatic Server is done via the web-based interface when the Phoromatic Server is active.</p>
+<p>The Phoromatic Server utilizes PHP/HHVM's built-in web-server capabilities and there's also a Phoronix Test Suite built-in WebSocket server that's also initiated for back-end processing. At this time there are no ports set by default for these services but must be defined within the user configuration file. With the Avahi zero-conf network discovery and other automated detection in place, there's little restrictions over the port selection.</p>
+<p>Systemd and Upstart service files are shipped with the Phoronix Test Suite for those that wish to have the services automatically run as daemons. The only new requirements over the basic Phoronix Test Suite system requirements is having PHP-SQLite support installed and the newer version of PHP or HHVM is recommended for offering the best support.</p>
+<h2>Example Deployments</h2>
+<h3>Use Case A: Unrestricted Internet Access, Local Result Storage</h3>
+<p>Systems on your network with unrestricted Internet access is the easiest and simplest deployment for the Phoronix Test Suite and Phoromatic. After installing the Phoronix Test Suite on the system you wish to designate the Phoromatic Server and have configured the <em>user-config.xml</em> file, simply run:</p>
+<p><strong>$ phoronix-test-suite start-phoromatic-server</strong></p>
+<p>Assuming you have no firewall or permission issues, the built-in web server and WebSocket server should proceed to initiate along with outputting the IP/port information for these services. Unless otherwise disabled from the user configuration file and if avahi-tools is present, the Phoromatic Server will be advertised with Avahi for zero-configuration networking.</p>
+<p>From the Phoromatic web interface you are able to create an account and from there proceed with the creating of test schedules, updating settings, and connecting systems. From the "client systems" you wish to use as the benchmarking nodes, it's simply a matter of running <strong>phoronix-test-suite phoromatic.connect</strong> with zero-conf networking or otherwise follow the information from the Phoromatic web interface for manual setup with the IP/port information.</p>
+<h3>Use Case B: No Internet Available To Client Systems</h3>
+<p>It's possible to run the Phoronix Test Suite and Phoromatic Server without a persistent Internet connection as long as you are able to first download the necessary files to the Phoromatic Server. After installing the Phoronix Test Suite on the system you wish to designate the Phoromatic Server and have configured the <em>user-config.xml</em> file, a few commands from the system while having an Internet connection will be able to cache the needed data:</p>
+<p><strong>$ phoronix-test-suite make-download-cache x264 xonotic ffmpeg</strong></p>
+<p>This command will simply download all of the needed test files for the tests/suites passed to the sub-command. Alternatively you could also pass pts/all to cache all tests. It's important though to just cache the tests/suites you'll be using on your network. This will generate the test file download cache by default to <em>~/.phoronix-test-suite/download-cache/</em> or <em>/usr/share/phoronix-test-suite/download-cache/</em> depending upon your write permissions. You can always run this command later with more test files. Alternatively, if you already have a number of tests installed on the system, simply running "phoronix-test-suite make-download-cache" will generate the cache based upon the currently installed tests.</p>
+<p><strong>$ phoronix-test-suite make-openbenchmarking-cache</strong></p>
+<p>This command will cache as much of the OpenBenchmarking.org meta-data as possible for test profiles and test suites. After the above commands, the Phoromatic Server should no longer need a persistent Internet connection.</p>
+<p><strong>$ phororonix-test-suite start-phoromatic-server</strong></p>
+<p>Proceed to start the Phoromatic Server and operate as normal.</p>
+<p>For the test clients without an Internet connection, as long as they're able to reach the Phoromatic Server, the Phoromatic Server should be able to automatically serve all of the needed test files download cache and OpenBenchmarking.org meta-data to the systems locally.</p>
+<h3>Use Case C: Phoromatic Across The Internet</h3>
+<p>If wishing to use the same Phoromatic Server across multiple geographic locations, it's easily possible -- you just lose out on the zero-conf networking ability. To let the Phoronix Test Suite client systems know about the remote Phoromatic Server, simply add the Phoromatic Server information to the client's <em>PhoromaticServers</em> element within the <em>user-config.xml</em>. Of course, make sure the Phoromatic Server has a globally resolvable IP address and its Phoromatic HTTP/WebSocket ports are open. Once informing the client of the Phoromatic Server, the use cases as above apply in the same manner.</p>
+<h2>Client Setup</h2>
+<p>From Phoronix Test Suite client systems running on the LAN, the following command will report all available detected Phoromatic Servers along with important server and debugging information:</p>
+<p><strong>$ phoronix-test-suite phoromatic.explore</strong></p>
+<p>With the following example output on finding one successful server:</p>
+<p><em>IP: 192.168.1.211<br />
+HTTP PORT: 5447<br />
+WEBSOCKET PORT: 5427<br />
+SERVER: PHP 5.5.9-1ubuntu4.4 Development Server<br />
+PHORONIX TEST SUITE: Phoronix Test Suite v5.4.0m1 [5313]<br />
+DOWNLOAD CACHE: 19 FILES / 2390 MB CACHE SIZE<br />
+SUPPORTED OPENBENCHMARKING.ORG REPOSITORIES:<br />
+      pts - Last Generated: 05 Oct 2014 07:16</em></p>
+<p>Phoromatic Servers are detected by the Phoronix Test Suite through Avahi or if manually configuring the Phoronix Test Suite clients to point to Phoromatic Servers. For networks without Avahi/auto-discovery support or for test systems that may be connecting from another network, the IP address and HTTP port number can be added to the local system's <em>~/.phoronix-test-suite/user-config.xml</em> with the <em>PhoromaticServers</em> element. Adding the <em>IP:port</em> (the Phoromatic Server's HTTP port) to the PhoromaticServers <em>user-config.xml</em> element for will perform targeted probing by the Phoronix Test Suite without any dependence on Avahi. Multiple Phoromatic Servers can be added if each IP:port is delimited by a comma.</p>
+<p>To connect a Phoronix Test Suite system for benchmarking to an account, log into your Phoromatic account from the web-interface and on the main/system pages will be instructions along with a specially formed string to run, e.g. <em>phoronix-test-suite phoromatic.connect 192.168.1.211:5447/I0SSJY</em>. When running that command once on the system(s) to be synced to that account, as the administrator you'll be able to validate/approve the systems from the Phoromatic web interface. After that, whenever the system(s) are to be running benchmarks, simply have the <strong>phoronix-test-suite phoromatic.connect</strong> command running on the system (after the initial account has been synced, simply running <strong>phoronix-test-suite phoromatic.connect</strong> is enough for the system to find the server and its account).</p>
+<h2>Root Administrator</h2>
+<p>The root administrator account is able to manage the server-level settings, e.g. Phoromatic storage location and other global settings related to the Phoronix Test Suite / Phoromatic Server, from the web user-interface.</p>
+<p>To enable the root administrator log-in, first from the server's command-line interface run <strong>phoronix-test-suite phoromatic.set-root-admin-password</strong> to set the password. Following that, you can log into the root administrator account via the web interface via the <em>rootadmin</em> user-name and the set password.</p>
+<h2>Other Advice</h2>
+<h3>Disable Internet Precaution</h3>
+<p>If you have an Internet connection but want to ensure your Phoronix Test Suite client doesn't attempt to use it for any matter, via the <em>~/.phoronix-test-suite/user-config.xml</em> you can set <em>NoInternetCommunication</em> to <em>TRUE</em>. There's also a NoNetworkCommunication tag, but setting that to TRUE will disable any form of network communication -- including communication with the Phoromatic Server.</p>
+<h3>Ports / Services</h3>
+<p>The Phoromatic Server process currently relies upon a PHP/HHVM built-in web server process and a PTS-hosted WebSocket server. The web server process handles the web UI and much of the responsibilities of the Phoromatic Server. Over time the PTS WebSocket server will be increasingly utilized for bi-directional, real-time communication between the server and clients -- including for features like viewing real-time hardware sensors of client systems from the server UI.</p>
+<h3>Systemd / Upstart</h3>
+<p>Packaged with the Phoronix Test Suite are basic <em>phoromatic-client</em> and <em>phoromatic-server</em> configurations for both Upstart and systemd init systems. The <em>phoromatic-server</em> configuration will launch the Phoronix Test Suite's Phoromatic Server and the <em>phoromatic-client</em> service will attempt to connect to a <u>pre-configured</u> Phoromatic Server. The systemd service files will automatically be installed via the Phoronix Test Suite <em>install-sh</em> process while the Upstart jobs can be copied from <em>deploy/phoromatic-upstart/*</em> to <em>/etc/init</em>.</p>
+<h3>Cache Verification</h3>
+<p>To confirm the files accessible to Phoronix Test Suite client systems, from the Phoromatic Server web user-interface go to the <em>settings</em> page followed by the <em>cache settings</em> link to view information about the download and OpenBenchmarking.org caches. From the client systems, running <strong>phoronix-test-suite phoromatic.explore</strong> will also supply cache statistics.</p>
+<h3>Log Files</h3>
+<p>The Phoromatic Server will produce a log file of events / debugging information to <em>~/.phoronix-test-suite/phoromatic.log</em> or <em>/var/log/phoromatic.log</em> depending upon the service's permissions. When running the Phoronix Test Suite Phoromatic client, the log will be written to one of the respective locations in <em>phoronix-test-suite.log</em>.</p>
+<h3>Multi-User Accounts</h3>
+<p>For each time a user account is made from the Phoromatic web UI's log-in page, all of the test schedules, systems, and other account information is separate to allow for a completely isolated multi-user system. If a main administrator (the one creating the account) wishes to have multiple users sharing the same account data, that user can create additional accounts from the <em>Users</em> tab of their account. The main administrator can make an additional administrator account or a &quot;viewer&quot; account that can consume the account's data but not create/modify the schedules, systems, or other account details.</p>
+<h3>
+<h3>File Locations</h3>
+<p>When running the Phoronix Test Suite Phoromatic Server as root, rather than using the <em>~/.phoronix-test-suite/</em> directory, the standard Linux file-system hierarchy standard is honored. The main storage path is <em>/var/lib/phoronix-test-suite/</em>, the user configuration file is <em>/etc/phoronix-test-suite.xml</em>, and <em>/var/cache/phoronix-test-suite/</em> for cache files.</p>
+<h3>Uploading Other Test Results</h3>
+<p>Unscheduled test results and other results found on connected systems to a Phoromatic account can upload the data to the Phoromatic Server using the <em>phoronix-test-suite phoromatic.upload-result &lt;result file identifier &gt;</em> sub-command.</p>
+<h3>User Context File Logging</h3>
+<p>For those utilizing custom set context script files as part of the Phoromatic test schedule, any important notes / log information can be written to the file specified by the <em>PHOROMATIC_LOG_FILE</em> environment variable set while running the user context scripts. The contents of that file is then sent to the Phoromatic Server otherwise the standard output of the script's execution is submitted to the Phoromatic Server for logging. These logs can then be viewed by the Phoromatic Server along with the test results. Other environment variables accessible when running a user context script include <em>PHOROMATIC_TRIGGER</em>, <em>PHOROMATIC_SCHEDULE_ID</em>, and <em>PHOROMATIC_SCHEDULE_PROCESS</em>.</p>
+</body>
+</html>

documentation/phoromatic.md

@@ -0,0 +1,92 @@
+
+# Phoronix Test Suite Phoromatic
+
+## Phoromatic Server
+
+### Introduction
+Phoromatic is a remote management system for the Phoronix Test Suite. Phoromatic allows the automatic (hence the name *Phoro-matic* ) scheduling of tests, remote installation of new tests, and the management of multiple test systems all through an intuitive, easy-to-use web interface. Tests can be scheduled to automatically run on a routine basis across multiple test systems. The test results are then available from this central, secure location.
+Phoromatic was originally introduced with Phoronix Test Suite 2.0 via Phoromatic.com as a project going back to 2008~2009. Phoromatic.com debuted as a hosted instance with the option of behind-the-firewall licensing for use within organizations. With Phoronix Test Suite 5.2 the model shifted to offer a local, open-source version of Phoromatic built into the Phoronix Test Suite code-base. Thanks to continued enterprise development, with Phoronix Test Suite 5.4 is now a fully-functioning, built-in version of Phoromatic that's open-source and can be used for behind-the-firewall testing without needing to push results to OpenBenchmarking.org and the ability to keep all results private.
+Phoromatic in Phoronix Test Suite 5.4 also has the ability to support zero-conf  network discovery using Avahi and the automatic distribution of needed test profiles/suites and test files. Phoronix Test Suite 5.4's Phoromatic is a significant breakthrough for open-source testing particularly those running this GPL benchmarking software within test labs and other large organizations.
+
+### Features
+Built atop the Phoronix Test Suite, Phoromatic offers many features for both enterprise and community/personal users:
+#### Automated Scheduling
+Whether it is every evening at 6:00PM, once every Thursday at 10:00AM or somewhere in between, Phoromatic can schedule tests to be run at user-defined intervals. The testing schedules can be updated through Phoromatic web interface. After the test(s) have run, the results will be immediately uploaded to Phoromatic.
+#### Extensible
+Any test profile or test suite that is compliant with the Phoronix Test Suite specification will work with Phoromatic. Phoromatic is able to leverage the hundreds of test profiles and test suites currently in the Phoronix Test Suite via OpenBenchmarking.org, along with any custom or proprietary test profiles you or your company utilize. Additionally, the Phoromatic interface allows the user to construct their own test suite(s).
+#### Remote Testing
+Once the test system is setup, all testing and management of that system can be done remotely. There is no need to execute Phoronix Test Suite commands locally using the GUI or command line version, but instead nearly all of the same features are accessible from the Phoromatic interface.
+#### Multi-System Support
+A single Phoromatic account is able to manage multiple test systems running the Phoronix Test Suite. Phoromatic supports grouping together test systems, tagging, and other features to support effectively managing many test systems. From the Phoromatic interface, installed system hardware and software from a given system can also be viewed.
+#### Turn-Key Deployment
+No additional software needs to be installed to support Phoromatic; all that's needed is Phoronix Test Suite 5.4 or later for full compatibility. New test systems can easily be synced with a given Phoromatic account by running a single command from the Phoronix Test Suite client.
+#### Result Management
+Test results are automatically uploaded to the Phoromatic account and remain private unless you opt to upload them to OpenBenchmarking.org. From the Phoromatic interface, results from multiple test systems can easily be compared and multiple results from the same systems can be used to track performance over time. There are also options to look at the statistical significance of the results and other features to efficiently and effectively analyze the system's performance.
+#### Decentralized
+Once the Phoronix Test Suite running on the Phoromatic Server has been able to cache all of the OpenBenchmarking.org test files and the needed files for each test, Phoromatic with any Phoronix Test Suite clients on your LAN can run fully decentralized without the need for a constant stream of OpenBenchmarking.org communication or Internet connection for that matter. (The only exception would be if your local systems don't have all their needed external dependencies and your system's package manager would need to install components like a compiler or necessary system libraries.
+#### Fully Open-Source
+Phoromatic is now fully open-source within the Phoronix Test Suite code-base for fostering greater development and new capabilities. Patches are welcome and Phoronix Media is available to provide commercial support and custom engineering services around Phoromatic and the Phoronix Test Suite.
+
+### Phoromatic Server Setup
+Phoromatic is built into the Phoronix Test Suite code-base and should be found in all packaged versions of the **phoronix-test-suite** . Starting the Phoromatic Server entails running phoronix-test-suite start-phoromatic-server after configuring the server information within *~/.phoronix-test-suite/user-config.xml* . The Phoromatic Server can with or without root permissions depending upon your firewall and the port numbers you wish to use for the server.
+On the "client side", any up-to-date version of the Phoronix Test Suite can automatically communicate with the Phoromatic Server. If Avahi support is available (commonly in Linux distribution repositories as _avahi-tools_ ), there should be zero-conf discovery if the Phoromatic Server and client systems are on the same LAN. If a Phoronix Test Suite client discovers a Phoromatic Server, it will attempt to use it automatically as a local download cache. In the event of no Internet connection, it will also attempt to obtain the needed OpenBenchmarking.org test/suite meta-data from the Phoromatic Server based upon its archived meta-data. This allows the Phoronix Test Suite / Phoromatic deployment on the LAN to be self-sustaining without an Internet connection as long as the systems have all installed test dependencies.
+Further configuration of the setup parameters for the Phoromatic Server and Phoronix Test Suite clients can be tuned via the *~/.phoronix-test-suite/user-config.xml* file. All control and configuration of the Phoromatic Server is done via the web-based interface when the Phoromatic Server is active.
+The Phoromatic Server utilizes PHP/HHVM's built-in web-server capabilities and there's also a Phoronix Test Suite built-in WebSocket server that's also initiated for back-end processing. At this time there are no ports set by default for these services but must be defined within the user configuration file. With the Avahi zero-conf network discovery and other automated detection in place, there's little restrictions over the port selection.
+Systemd and Upstart service files are shipped with the Phoronix Test Suite for those that wish to have the services automatically run as daemons. The only new requirements over the basic Phoronix Test Suite system requirements is having PHP-SQLite support installed and the newer version of PHP or HHVM is recommended for offering the best support.
+
+### Example Deployments
+#### Use Case A: Unrestricted Internet Access, Local Result Storage
+Systems on your network with unrestricted Internet access is the easiest and simplest deployment for the Phoronix Test Suite and Phoromatic. After installing the Phoronix Test Suite on the system you wish to designate the Phoromatic Server and have configured the *user-config.xml* file, simply run:
+**$ phoronix-test-suite start-phoromatic-server**
+Assuming you have no firewall or permission issues, the built-in web server and WebSocket server should proceed to initiate along with outputting the IP/port information for these services. Unless otherwise disabled from the user configuration file and if avahi-tools is present, the Phoromatic Server will be advertised with Avahi for zero-configuration networking.
+From the Phoromatic web interface you are able to create an account and from there proceed with the creating of test schedules, updating settings, and connecting systems. From the "client systems" you wish to use as the benchmarking nodes, it's simply a matter of running **phoronix-test-suite phoromatic.connect** with zero-conf networking or otherwise follow the information from the Phoromatic web interface for manual setup with the IP/port information.
+#### Use Case B: No Internet Available To Client Systems
+It's possible to run the Phoronix Test Suite and Phoromatic Server without a persistent Internet connection as long as you are able to first download the necessary files to the Phoromatic Server. After installing the Phoronix Test Suite on the system you wish to designate the Phoromatic Server and have configured the *user-config.xml* file, a few commands from the system while having an Internet connection will be able to cache the needed data:
+**$ phoronix-test-suite make-download-cache x264 xonotic ffmpeg**
+This command will simply download all of the needed test files for the tests/suites passed to the sub-command. Alternatively you could also pass pts/all to cache all tests. It's important though to just cache the tests/suites you'll be using on your network. This will generate the test file download cache by default to *~/.phoronix-test-suite/download-cache/* or */usr/share/phoronix-test-suite/download-cache/* depending upon your write permissions. You can always run this command later with more test files. Alternatively, if you already have a number of tests installed on the system, simply running "phoronix-test-suite make-download-cache" will generate the cache based upon the currently installed tests.
+**$ phoronix-test-suite make-openbenchmarking-cache**
+This command will cache as much of the OpenBenchmarking.org meta-data as possible for test profiles and test suites. After the above commands, the Phoromatic Server should no longer need a persistent Internet connection.
+**$ phororonix-test-suite start-phoromatic-server**
+Proceed to start the Phoromatic Server and operate as normal.
+For the test clients without an Internet connection, as long as they're able to reach the Phoromatic Server, the Phoromatic Server should be able to automatically serve all of the needed test files download cache and OpenBenchmarking.org meta-data to the systems locally.
+#### Use Case C: Phoromatic Across The Internet
+If wishing to use the same Phoromatic Server across multiple geographic locations, it's easily possible -- you just lose out on the zero-conf networking ability. To let the Phoronix Test Suite client systems know about the remote Phoromatic Server, simply add the Phoromatic Server information to the client's *PhoromaticServers* element within the *user-config.xml* . Of course, make sure the Phoromatic Server has a globally resolvable IP address and its Phoromatic HTTP/WebSocket ports are open. Once informing the client of the Phoromatic Server, the use cases as above apply in the same manner.
+
+### Client Setup
+From Phoronix Test Suite client systems running on the LAN, the following command will report all available detected Phoromatic Servers along with important server and debugging information:
+**$ phoronix-test-suite phoromatic.explore**
+With the following example output on finding one successful server:
+*IP: 192.168.1.211
+HTTP PORT: 5447
+WEBSOCKET PORT: 5427
+SERVER: PHP 5.5.9-1ubuntu4.4 Development Server
+PHORONIX TEST SUITE: Phoronix Test Suite v5.4.0m1 [5313]
+DOWNLOAD CACHE: 19 FILES / 2390 MB CACHE SIZE
+SUPPORTED OPENBENCHMARKING.ORG REPOSITORIES:
+      pts - Last Generated: 05 Oct 2014 07:16*
+Phoromatic Servers are detected by the Phoronix Test Suite through Avahi or if manually configuring the Phoronix Test Suite clients to point to Phoromatic Servers. For networks without Avahi/auto-discovery support or for test systems that may be connecting from another network, the IP address and HTTP port number can be added to the local system's *~/.phoronix-test-suite/user-config.xml* with the *PhoromaticServers* element. Adding the *IP:port* (the Phoromatic Server's HTTP port) to the PhoromaticServers *user-config.xml* element for will perform targeted probing by the Phoronix Test Suite without any dependence on Avahi. Multiple Phoromatic Servers can be added if each IP:port is delimited by a comma.
+To connect a Phoronix Test Suite system for benchmarking to an account, log into your Phoromatic account from the web-interface and on the main/system pages will be instructions along with a specially formed string to run, e.g. *phoronix-test-suite phoromatic.connect 192.168.1.211:5447/I0SSJY* . When running that command once on the system(s) to be synced to that account, as the administrator you'll be able to validate/approve the systems from the Phoromatic web interface. After that, whenever the system(s) are to be running benchmarks, simply have the **phoronix-test-suite phoromatic.connect** command running on the system (after the initial account has been synced, simply running **phoronix-test-suite phoromatic.connect** is enough for the system to find the server and its account).
+
+### Root Administrator
+The root administrator account is able to manage the server-level settings, e.g. Phoromatic storage location and other global settings related to the Phoronix Test Suite / Phoromatic Server, from the web user-interface.
+To enable the root administrator log-in, first from the server's command-line interface run **phoronix-test-suite phoromatic.set-root-admin-password** to set the password. Following that, you can log into the root administrator account via the web interface via the *rootadmin* user-name and the set password.
+
+### Other Advice
+#### Disable Internet Precaution
+If you have an Internet connection but want to ensure your Phoronix Test Suite client doesn't attempt to use it for any matter, via the *~/.phoronix-test-suite/user-config.xml* you can set *NoInternetCommunication* to *TRUE* . There's also a NoNetworkCommunication tag, but setting that to TRUE will disable any form of network communication -- including communication with the Phoromatic Server.
+#### Ports / Services
+The Phoromatic Server process currently relies upon a PHP/HHVM built-in web server process and a PTS-hosted WebSocket server. The web server process handles the web UI and much of the responsibilities of the Phoromatic Server. Over time the PTS WebSocket server will be increasingly utilized for bi-directional, real-time communication between the server and clients -- including for features like viewing real-time hardware sensors of client systems from the server UI.
+#### Systemd / Upstart
+Packaged with the Phoronix Test Suite are basic *phoromatic-client* and *phoromatic-server* configurations for both Upstart and systemd init systems. The *phoromatic-server* configuration will launch the Phoronix Test Suite's Phoromatic Server and the *phoromatic-client* service will attempt to connect to a _pre-configured_ Phoromatic Server. The systemd service files will automatically be installed via the Phoronix Test Suite *install-sh* process while the Upstart jobs can be copied from *deploy/phoromatic-upstart/** to */etc/init* .
+#### Cache Verification
+To confirm the files accessible to Phoronix Test Suite client systems, from the Phoromatic Server web user-interface go to the *settings* page followed by the *cache settings* link to view information about the download and OpenBenchmarking.org caches. From the client systems, running **phoronix-test-suite phoromatic.explore** will also supply cache statistics.
+#### Log Files
+The Phoromatic Server will produce a log file of events / debugging information to *~/.phoronix-test-suite/phoromatic.log* or */var/log/phoromatic.log* depending upon the service's permissions. When running the Phoronix Test Suite Phoromatic client, the log will be written to one of the respective locations in *phoronix-test-suite.log* .
+#### Multi-User Accounts
+For each time a user account is made from the Phoromatic web UI's log-in page, all of the test schedules, systems, and other account information is separate to allow for a completely isolated multi-user system. If a main administrator (the one creating the account) wishes to have multiple users sharing the same account data, that user can create additional accounts from the *Users* tab of their account. The main administrator can make an additional administrator account or a "viewer" account that can consume the account's data but not create/modify the schedules, systems, or other account details.
+#### File Locations
+When running the Phoronix Test Suite Phoromatic Server as root, rather than using the *~/.phoronix-test-suite/* directory, the standard Linux file-system hierarchy standard is honored. The main storage path is */var/lib/phoronix-test-suite/* , the user configuration file is */etc/phoronix-test-suite.xml* , and */var/cache/phoronix-test-suite/* for cache files.
+#### Uploading Other Test Results
+Unscheduled test results and other results found on connected systems to a Phoromatic account can upload the data to the Phoromatic Server using the *phoronix-test-suite phoromatic.upload-result <result file identifier >* sub-command.
+#### User Context File Logging
+For those utilizing custom set context script files as part of the Phoromatic test schedule, any important notes / log information can be written to the file specified by the *PHOROMATIC_LOG_FILE* environment variable set while running the user context scripts. The contents of that file is then sent to the Phoromatic Server otherwise the standard output of the script's execution is submitted to the Phoromatic Server for logging. These logs can then be viewed by the Phoromatic Server along with the test results. Other environment variables accessible when running a user context script include *PHOROMATIC_TRIGGER* , *PHOROMATIC_SCHEDULE_ID* , and *PHOROMATIC_SCHEDULE_PROCESS* .

documentation/phoronix-test-suite-window.md

@@ -0,0 +1,27 @@
+
+# Phoronix Test Suite On Windows
+
+### Introduction
+Phoronix Test Suite 8.0 features rewritten Windows support that is at a near feature parity to the program's long-standing support for Linux, macOS, BSD and Solaris operating systems. To make it abundantly clear, if you are using a Phoronix Test Suite version pre-8.0, you are best upgrading or ideally using Phoronix Test Suite Git as the Windows support remains in very active development at the moment as of early 2018.
+The Phoronix Test Suite Windows support currently targets **Windows 10 x64** and **Windows Server 2016 x64** . Earlier versions of Windows, namely Windows Server 2012 and Windows 8, may work to some extent but some hardware/software reporting features and other capabilities may be missing or report warning messages. The Phoronix Test Suite Windows support is also exclusively focused on x86 64-bit support: the Phoronix Test Suite itself will run on x86 32-bit but many of the program dependencies are configured for making use of 64-bit binaries.
+
+### Windows Setup / Dependencies
+As with Phoronix Test Suite on Linux and other operating systems, the principal dependency is on PHP (PHP v5.3 or newer, including PHP 7.x). Running the *phoronix-test-suite.bat* file launcher for the Phoronix Test Suite on Windows will attempt to download and setup PHP on the system under *C:\PHP* as the default location should PHP support not be found within your system's *Program Files* directories. The PHP Windows build does depend upon Microsoft Visual C++ redistributable libraries, which the Windows launcher will also attempt to download and install if needed.
+The Phoronix Test Suite on Windows does depend upon [Cygwin](https://www.cygwin.com/) for its Bash interpreter and other basic utilities to ease the process of porting test profiles to Windows with being able to use many of the same test installation scripts on Windows/Linux/macOS/BSD/Solaris then largely unmodified. Most of the Windows tests depend upon their respective native Windows applications/binaries while this Cygwin support is a convenience for handling these Bash setup scripts and also some test profiles that depend upon a GNU toolchain. The Phoronix Test Suite will attempt to download and setup Cygwin on the system if Cygwin isn't found in its default location of *C:\cygwin64* .
+Various test profiles may depend upon other "external dependencies" like Python, PERL, Steam, and Java, as examples. The Phoronix Test Suite as with its support for other operating systems and Linux distributions will attempt to install these needed dependencies on a per-test basis when needed if existing support is not detected on the system.
+
+### Running The Phoronix Test Suite On Windows
+The Phoronix Test Suite can run from its local directory and does not need to be "installed" to a system path or any other "setup" process prior to execution. On a clean install of Windows 10 x64 or Windows Server 2016, deploying the Phoronix Test Suite is designed to be as easy and straight-forward as possible:
+1. Download the Phoronix Test Suite 8.0+ or [Phoronix-Test-Suite from GitHub](https://github.com/phoronix-test-suite/phoronix-test-suite) ( [zip file](https://github.com/phoronix-test-suite/phoronix-test-suite/archive/master.zip) ).
+2. From the Command Prompt or PowerShell, enter the *phoronix-test-suite* directory whether it be from Git or a zipped download.
+3. Run the *phoronix-test-suite.bat* file that should proceed to run the Phoronix Test Suite just as you would on any other operating system. If needed the Phoronix Test Suite will try to initially download and setup PHP if needed followed by the attempted automatic Cygwin setup, etc.
+4. Any of the Phoronix Test Suite commands from other operating systems should work on Windows. If you are new to the Phoronix Test Suite, you may enjoy a bit more guided experience by running the **phoronix-test-suite shell** command.
+
+### Test Profiles On Windows
+As of March 2018, around 50 of the test profiles are currently compatible with the Phoronix Test Suite on Windows. This includes many of the popular benchmarks and other interesting test cases. Over time more test profiles will continue to be ported to Windows where applicable and there are also some Windows-only tests also supported for execution by the Phoronix Test Suite.
+
+### Getting Started
+Besides **phoronix-test-suite shell** and **phoronix-test-suite help** , there is also **phoronix-test-suite interactive** for helping new users understand Phoronix Test Suite benchmarking. Long story short, it should be as easy as running **phoronix-test-suite benchmark c-ray** or **phoronix-test-suite benchmark crafty** as some examples for carrying out automated, cross-platform benchmarks in a side-by-side and fully-reproducible manner.
+
+### Support
+Community technical support is available via [GitHub](https://github.com/phoronix-test-suite/phoronix-test-suite/issues) or general inquiries via [the Phoronix Forums](https://www.phoronix.com/forums/forum/phoronix/phoronix-test-suite) . For enterprise inquiries, commercial support, and custom engineering services, [contact us](http://phoronix-test-suite.com/?k=commercial) .

documentation/phoronix-test-suite-windows.html

@@ -0,0 +1,27 @@
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+<Title>Phoronix Test Suite On Windows</Title>
+</head>
+<body>
+
+<h2>Introduction</h2>
+<p>Phoronix Test Suite 8.0 features rewritten Windows support that is at a near feature parity to the program's long-standing support for Linux, macOS, BSD and Solaris operating systems. To make it abundantly clear, if you are using a Phoronix Test Suite version pre-8.0, you are best upgrading or ideally using Phoronix Test Suite Git as the Windows support remains in very active development at the moment as of early 2018.</p>
+<p>The Phoronix Test Suite Windows support currently targets <strong>Windows 10 x64</strong> and <strong>Windows Server 2016 x64</strong>. Earlier versions of Windows, namely Windows Server 2012 and Windows 8, may work to some extent but some hardware/software reporting features and other capabilities may be missing or report warning messages. The Phoronix Test Suite Windows support is also exclusively focused on x86 64-bit support: the Phoronix Test Suite itself will run on x86 32-bit but many of the program dependencies are configured for making use of 64-bit binaries.</p>
+<h2>Windows Setup / Dependencies</h2>
+<p>As with Phoronix Test Suite on Linux and other operating systems, the principal dependency is on PHP (PHP v5.3 or newer, including PHP 7.x). Running the <em>phoronix-test-suite.bat</em> file launcher for the Phoronix Test Suite on Windows will attempt to download and setup PHP on the system under <em>C:\PHP</em> as the default location should PHP support not be found within your system's <em>Program Files</em> directories. The PHP Windows build does depend upon Microsoft Visual C++ redistributable libraries, which the Windows launcher will also attempt to download and install if needed.</p>
+<p>The Phoronix Test Suite on Windows does depend upon <a href="https://www.cygwin.com/">Cygwin</a> for its Bash interpreter and other basic utilities to ease the process of porting test profiles to Windows with being able to use many of the same test installation scripts on Windows/Linux/macOS/BSD/Solaris then largely unmodified. Most of the Windows tests depend upon their respective native Windows applications/binaries while this Cygwin support is a convenience for handling these Bash setup scripts and also some test profiles that depend upon a GNU toolchain. The Phoronix Test Suite will attempt to download and setup Cygwin on the system if Cygwin isn't found in its default location of <em>C:\cygwin64</em>.</p>
+<p>Various test profiles may depend upon other &quot;external dependencies&quot; like Python, PERL, Steam, and Java, as examples. The Phoronix Test Suite as with its support for other operating systems and Linux distributions will attempt to install these needed dependencies on a per-test basis when needed if existing support is not detected on the system.</p>
+<h2>Running The Phoronix Test Suite On Windows</h2>
+<p>The Phoronix Test Suite can run from its local directory and does not need to be &quot;installed&quot; to a system path or any other &quot;setup&quot; process prior to execution. On a clean install of Windows 10 x64 or Windows Server 2016, deploying the Phoronix Test Suite is designed to be as easy and straight-forward as possible:</p>
+<p>1. Download the Phoronix Test Suite 8.0+ or <a href="https://github.com/phoronix-test-suite/phoronix-test-suite">Phoronix-Test-Suite from GitHub</a> (<a href="https://github.com/phoronix-test-suite/phoronix-test-suite/archive/master.zip">zip file</a>).</p>
+<p>2. From the Command Prompt or PowerShell, enter the <em>phoronix-test-suite</em> directory whether it be from Git or a zipped download.</p>
+<p>3. Run the <em>phoronix-test-suite.bat</em> file that should proceed to run the Phoronix Test Suite just as you would on any other operating system. If needed the Phoronix Test Suite will try to initially download and setup PHP if needed followed by the attempted automatic Cygwin setup, etc.</p>
+<p>4. Any of the Phoronix Test Suite commands from other operating systems should work on Windows. If you are new to the Phoronix Test Suite, you may enjoy a bit more guided experience by running the <strong>phoronix-test-suite shell</strong> command.</p>
+<h2>Test Profiles On Windows</h2>
+<p>As of March 2018, around 50 of the test profiles are currently compatible with the Phoronix Test Suite on Windows. This includes many of the popular benchmarks and other interesting test cases. Over time more test profiles will continue to be ported to Windows where applicable and there are also some Windows-only tests also supported for execution by the Phoronix Test Suite.</p>
+<h2>Getting Started</h2>
+<p>Besides <strong>phoronix-test-suite shell</strong> and <strong>phoronix-test-suite help</strong>, there is also <strong>phoronix-test-suite interactive</strong> for helping new users understand Phoronix Test Suite benchmarking. Long story short, it should be as easy as running <strong>phoronix-test-suite benchmark c-ray</strong> or <strong>phoronix-test-suite benchmark crafty</strong> as some examples for carrying out automated, cross-platform benchmarks in a side-by-side and fully-reproducible manner.</p>
+<h2>Support</h2>
+<p>Community technical support is available via <a href="https://github.com/phoronix-test-suite/phoronix-test-suite/issues">GitHub</a> or general inquiries via <a href="https://www.phoronix.com/forums/forum/phoronix/phoronix-test-suite">the Phoronix Forums</a>. For enterprise inquiries, commercial support, and custom engineering services, <a href="http://phoronix-test-suite.com/?k=commercial">contact us</a>.</p>
+</body>
+</html>

documentation/phoronix-test-suite.md

@@ -0,0 +1,787 @@
+
+# Getting Started
+
+## Overview
+The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available for Linux, Solaris, Mac OS X, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners. Version 1.0 of the Phoronix Test Suite was publicly released in 2008.
+The Phoronix Test Suite client itself is a test framework for providing seamless execution of test profiles and test suites. There are more than 200 tests available by default, which are transparently available via [OpenBenchmarking.org](http://openbenchmarking.org/) integration. Of these default test profiles there is a range of sub-systems that can be tested and a range of hardware from mobile devices to desktops and worksrtations/servers. New tests can be easily introduced via the Phoronix Test Suite's extensible test architecture, with test profiles consisting of XML files and shell scripts. Test profiles can produce a quantitative result or other qualitative/abstract results like image quality comparisons and pass/fail. Using Phoronix Test Suite modules, other data can also be automatically collected at run-time such as the system power consumption, disk usage, and other software/hardware sensors. Test suites contain references to test profiles to execute as part of a set or can also reference other test suites. Test suites are defined via an XML schema.
+Running the Phoronix Test Suite for the first time can be as simple as issuing a command such as *phoronix-test-suite benchmark c-ray* , which would proceed to install a simple CPU test, execute the test, and report the results. Along with the results, the system's hardware/software information is collected in a detailed manner, relevant system logs, and other important system attributes such as compiler flags and system state. Users can optionally upload their results to OpenBenchmarking.org for sharing results with others, comparing results against other systems, and to carry out further analysis.
+
+## OpenBenchmarking.org
+OpenBenchmarking.org is an open, collaborative testing platform that makes the Phoronix Test Suite an even more extensible platform for conducting automated tests with complete integration into Phoronix Test Suite test client. OpenBenchmarking.org serves as a repository for storing test profiles, test suites, and result data. Test profiles and suites are stored in the OpenBenchmarking.org cloud to allow for new/updated tests to be seamlessly obtained via the Phoronix Test Suite without needing to manually update the Phoronix Test Suite client. OpenBenchmarking.org also makes it easy to facilitate side-by-side comparisons with any other results stored in the OpenBenchmarking.org cloud. Any Phoronix Test Suite user is permitted to upload their test results, test profiles, and suites to OpenBenchmarking.org.
+When finding a set of results on OpenBenchmarking.org (e.g. [an example result file](http://openbenchmarking.org/result/1203160-BY-NVTEGRA3785) ), it's as easy as running the Phoronix Test Suite with that OpenBenchmarking.org ID to perform an automated side-by-side comparison (e.g. *phoronix-test-suite benchmark 1203160-BY-NVTEGRA3785* ).
+Thanks to the wealth of test data (results, system logs, etc) from crowd-sourced benchmarking via the Phoronix Test Suite, a plethora of analytical features are also available from OpenBenchmarking.org.
+
+## Phoromatic
+Phoromatic is a remote management system for the Phoronix Test Suite that allows the automatic scheduling of tests, remote installation of new tests, and the management of multiple test systems all through an intuitive, easy-to-use web interface. Tests can be scheduled to automatically run on a routine basis across multiple test systems. Phoromatic can also interface with revision control systems to offer support for issuing new tests on a context-basis, such as whenever a Git commit has been pushed or new daily image available. The test results are then available from this central, secure location.
+Phoromatic is an add-on to the Phoronix Test Suite that's primarily intended for enterprise users when facilitating tests across a wide-spectrum of hardware within a test lab or when needing to carry out tests on a routine basis.
+
+# User Options
+The following options are currently supported by the Phoronix Test Suite client. A list of available options can also be found by running *phoronix-test-suite help.*
+
+---
+
+## System
+#### diagnostics
+This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
+#### interactive
+A simple text-driven interactive interface to the Phoronix Test Suite.
+#### php-conf
+This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
+#### shell
+A simple text-driven shell interface / helper to the Phoronix Test Suite. Ideal for those that may be new to the Phoronix Test Suite
+#### system-info
+Display the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.
+#### system-sensors
+Display the installed system hardware and software sensors in real-time as detected by the Phoronix Test Suite Phodevi Library.
+
+## Test Installation
+#### force-install  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem or wishing to re-install test(s) due to compiler or other environmental changes.
+#### install  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option will install the selected test(s) inside the testing environment directory. The install process from downloading of the test files to the installation is fully automated. The install option needs to be supplied with the test name or suite as an argument. Optionally, a OpenBenchmarking.org ID or the name of a saved results file can be supplied as well and the test(s) to install will automatically be extracted from that information. If the test is already installed and was run by the latest version of the installation process, no action will be taken. Multiple arguments can be supplied to install additional tests at the same time.
+#### install-dependencies  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option will install the external dependencies needed by the selected test(s) using the distribution's package management system. For example, some tests depend upon GCC for compiling code. If GCC is not detected on the system, the Phoronix Test Suite will attempt to install GCC using the distribution's package management system. If you are running this command as a local user, you may be prompted for the root password while the process is running. For unsupported distributions, the dependency names will be displayed along with common names for the package. The install-dependencies option needs to be supplied with the test name or suite as an argument. When using the install option, the external dependencies are automatically checked.
+#### make-download-cache
+This option will create a download cache for use by the Phoronix Test Suite. The download cache is created of test files already downloaded to the local system. If passing any test/suite names to make-download-cache, the needed files for those test profiles will first be automatically downloaded before creating the cache.
+#### remove-installed-test  [Test]
+This option will permanently remove a installed test by the Phoronix Test Suite.
+
+## Testing
+#### auto-compare
+This option will autonomously determine the most relevant test(s) to run for any selected sub-system(s). The tests to run are determined via OpenBenchmarking.org integration with the global results pool. Related test results from OpenBenchmarking.org are also merged to provide a straight-forward and effective means of carrying out a system comparison. If wishing to find comparable results for any particular test profile(s), simply pass the test profile names as additional arguments to this command.
+#### benchmark  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option will install the selected test(s) (if needed) and will proceed to run the test(s). This option is equivalent to running phoronix-test-suite with the install option followed by the run option. Multiple arguments can be supplied to run additional tests at the same time and save the results into one file.
+#### estimate-run-time  [Test | Suite | OpenBenchmarking ID | Test Result]
+This option will provide estimates for test run-time / length.
+#### finish-run  [Test Result]
+This option can be used if a test run had not properly finished running all tests within a saved results file. Using this option when specifying a saved results file where all tests had not completed will attempt to finish testing on the remaining tests where there are missing results.
+#### run  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option will run the selected test(s).
+#### run-random-tests
+This option will query OpenBenchmarking.org to run random benchmarks and result comparisons on the system. This test can be used for simply supplying interesting results from your system onto OpenBenchmarking.org, stressing your system with random workloads, seeding new OpenBenchmarking.org results, etc. Basic options are provided at start-up for tuning the randomness of the testing when running this command.
+#### run-tests-in-suite
+This option can be used if you wish to run all of the tests found in a supplied suite, but you wish to re-configure each of the test options rather than using the defaults supplied by the suite.
+#### stress-run  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option will run the passed tests/suites in the multi-process stress-testing mode. The stress-run mode will not produce a result file but is rather intended for running multiple test profiles concurrently to stress / burn-in the system. The number of tests to run concurrently can be toggled via the PTS_CONCURRENT_TEST_RUNS environment variable and by default is set to a value of 2.
+
+## Batch Testing
+#### batch-benchmark  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option and its arguments are equivalent to the benchmark option, but the process will be run in the Phoronix Test Suite batch mode.
+#### batch-install  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+If you wish to run the install process in the Phoronix Test Suite batch mode but do not wish to run any tests at this time. Running the install process in the batch mode will use the default values and not prompt the user of any possible options, to ensure the process is fully automated.
+#### batch-run  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option and its arguments are equivalent to the run option, but the process will be run in the Phoronix Test Suite batch mode.
+#### batch-setup
+This option is used to configure the batch mode options for the Phoronix Test Suite, which is subsequently written to the user configuration file. Among the options are whether to automatically upload the test results to OpenBenchmarking.org and prompting for the saved file name.
+#### default-benchmark  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option will install the selected test(s) (if needed) and will proceed to run the test(s) in the defaults mode. This option is equivalent to running phoronix-test-suite with the install option followed by the default-run option.
+#### default-run  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option will run the selected test(s). The name of the test or suite must be supplied or the OpenBenchmarking.org ID or saved local file name. Multiple arguments can be supplied to run additional tests at the same time and save the results in a suite-like fashion. Unlike the normal run option, the default-run will not prompt the user to select from the available test options but will instead use the default options as automatically set by pts-core or the test profile. Use batch-run to automatically test all of the available options.
+#### dry-run  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option and its arguments pre-set the Phoronix Test Suite batch run mode with enforcing of defaults to not save any results and other behavior intended for a dry/test run. This option is primarily intended for testing/evaluation purposes.
+#### internal-run  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option and its arguments pre-set the Phoronix Test Suite batch run mode with sane values for carrying out benchmarks in a semi-automated manner and without uploading any of the result data to the public OpenBenchmarking.org.
+
+## OpenBenchmarking.org
+#### clone-result  [OpenBenchmarking ID]  ...
+This option will download a local copy of a file that was saved to OpenBenchmarking.org, as long as a valid public ID is supplied.
+#### list-recommended-tests
+This option will list recommended test profiles for benchmarking sorted by hardware sub-system. The recommended tests are determined via querying OpenBenchmarking.org and determining the most popular tests for a given environment based upon the number of times a test profile has been downloaded, the number of test results available on OpenBenchmarking.org for a given test profile, the age of the test profile, and other weighted factors.
+#### make-openbenchmarking-cache
+This option will attempt to cache the test profile/suite meta-data from OpenBenchmarking.org for all linked repositories. This is useful if you're going to be running the Phoronix Test Suite / Phoromatic behind a firewall or without any Internet connection. Those with unrestricted Internet access or not utilizing a large local deployment of the Phoronix Test Suite / Phoromatic shouldn't need to run this command.
+#### openbenchmarking-changes
+This option will list recent changes to test profiles of enabled OpenBenchmarking.org repositories.
+#### openbenchmarking-login
+This option is used for controlling your Phoronix Test Suite client options for OpenBechmarking.org and syncing the client to your account.
+#### openbenchmarking-refresh
+This option is used for refreshing the stored OpenBenchmarking.org repostory information and other data. The Phoronix Test Suite will automatically refresh this data every three days or when other thresholds are exceeded, but this command can be used to manually refresh/updates the data.
+#### openbenchmarking-repositories
+This option will list the OpenBenchmarking.org repositories currently linked to this Phoronix Test Suite client instance.
+#### openbenchmarking-uploads
+This option will list any recent test result uploads from the system's IP address to OpenBenchmarking.org.
+#### upload-result  [Test Result]
+This option is used for uploading a test result to OpenBenchmarking.org.
+#### upload-test-profile
+This option can be used for uploading a test profile to your account on OpenBenchmarking.org. By uploading your test profile to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution in a seamless manner by other Phoronix Test Suite clients.
+#### upload-test-suite
+This option can be used for uploading a test suite to your account on OpenBenchmarking.org. By uploading your test suite to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution.
+
+## Information
+#### info  [Test | Suite | OpenBenchmarking ID | Test Result]
+This option will show details about the supplied test, suite, virtual suite, or result file.
+#### list-all-tests
+This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories. Unlike the other test listing options, list-all-tests will show deprecated tests, potentially broken tests, or other tests not recommended for all environments. The only check in place is ensuring the test profiles are at least compatible with the operating system in use.
+#### list-available-suites
+This option will list all test suites that are available from the enabled OpenBenchmarking.org repositories.
+#### list-available-tests
+This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state. If the system has no Internet access, it will only list the test profiles where the necesary test assets are available locally on the system or on an available network cache (the same behavior as using the list-cached-tests sub-command), unless using the list-all-tests option to override this behavior.
+#### list-available-virtual-suites
+This option will list all available virtual test suites that can be dynamically created based upon the available tests from enabled OpenBenchmarking.org repositories.
+#### list-cached-tests
+This option will list all test profiles where any needed test profiles are already cached or available from the local system under test. This is primarily useful if testing offline/behind-the-firewall and other use-cases where wanting to rely only upon local data.
+#### list-installed-dependencies
+This option will list all of the packages / external test dependencies that are already installed on the system that the Phoronix Test Suite may potentially depend upon by test profiles.
+#### list-installed-suites
+This option will list all suites that are currently installed on the system.
+#### list-installed-tests
+This option will list all test profiles that are currently installed on the system.
+#### list-missing-dependencies
+This option will list all of the packages / external test dependencies that are missing from the system that the Phoronix Test Suite may potentially need by select test profiles.
+#### list-not-installed-tests
+This option will list all test profiles that are supported and available but presently NOT installed on the system.
+#### list-possible-dependencies
+This option will list all of the packages / external test dependencies that are are potentially used by the Phoronix Test Suite.
+#### list-saved-results
+This option will list all of the saved test results found on the system.
+#### list-test-usage
+This option will list various details about installed tests and their usage.
+#### list-unsupported-tests
+This option will list all available test profiles that are available from the enabled OpenBenchmarking.org repositories but are NOT SUPPORTED on the given hardware/software platform. This is mainly a debugging option for those looking for test profiles to potentially port to new platforms, etc.
+#### search
+This option provides command-line searching abilities for test profiles / test suites / test results. The search query can be passed as a parameter otherwise the user is prompted to input their search query..
+
+## Asset Creation
+#### build-suite
+This option will guide the user through the process of generating their own test suite, which they can then run. Optionally, passed as arguments can be the test(s) or suite(s) to add to the suite to be created, instead of being prompted through the process.
+#### create-test-profile
+This option can be used for creating a Phoronix Test Suite test profile by answering questions about the test for constructing the test profile XML meta-data and handling other boiler-plate basics for getting started in developing new tests.
+#### debug-benchmark  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option is intended for use by test profile writers and is identical to the <em>run</em> option but will yield more information during the run process that can be used to debug issues with a test profile or to verify the test profile is functioning correctly.
+#### debug-install  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option is intended for use by test profile writers and is identical to the install option but will yield more information during the run process that can be used to debug issues with a test profile installer or to verify the test profile is functioning correctly.
+#### debug-result-parser  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option is intended for use by test profile writers and is used for debugging a result parser. No test execution is done, but there must already be PTS-generated .log files present within the test's installation directory.
+#### debug-test-download-links  [Test | Suite | OpenBenchmarking ID | Test Result]
+This option will check all download links within the specified test profile(s) to ensure there are no broken URLs.
+#### download-test-files  [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This will download the selected test file(s) to the Phoronix Test Suite download cache but will not install the tests.
+#### inspect-test-profile  [Test]
+This option can be used for inspecting a Phoronix Test Suite test profile with providing inside details on test profiles for debugging / evaluation / learning purposes.
+#### result-file-to-suite  [Test Result]
+This option will guide the user through the process of generating their own test suite, which they can then run, that is based upon an existing test results file.
+#### validate-result-file
+This option can be used for validating a Phoronix Test Suite result file as being compliant against the OpenBenchmarking.org specification.
+#### validate-test-profile  [Test]
+This option can be used for validating a Phoronix Test Suite test profile as being compliant against the OpenBenchmarking.org specification.
+#### validate-test-suite
+This option can be used for validating a Phoronix Test Suite test suite as being compliant against the OpenBenchmarking.org specification.
+
+## Result Management
+#### analyze-all-runs  [Test Result]
+This option will generate a candlestick graph showing the distribution of results from all trial runs. The candlestick graph is similar to the Japanese candlestick charts used by the financial industry, except instead of representing stock data it is numerical result data from all trial runs.\n\nThe tip of the upper-wick represents the highest value of the test runs with the tip of the lower-wick representing the lowest value of all test runs. The upper-edge of the candle body represents the first or last run value and the lower-edge represents the first or last run value. Lastly, if the last run value is less than the first run value, the candle body is the same color as the graph background, otherwise the last run value is greater.
+#### auto-sort-result-file  [Test Result]
+This option is used if you wish to automatically attempt to sort the results by their result identifier string.
+#### edit-result-file  [Test Result]
+This option is used if you wish to edit the title and description of an existing result file.
+#### extract-from-result-file  [Test Result]
+This option will extract a single set of test results from a saved results file that contains multiple test results that have been merged. The user is the prompted to specify a new result file name and select which result identifier to extract.
+#### merge-results  [Test Result]  ...
+This option will manually merge multiple sets of test results generated by the Phoronix Test Suite.
+#### refresh-graphs  [Test Result]
+This option will re-render and save all result graphs within a saved file. This option can be used when making modifications to the graphing code or its color/option configuration file and testing the changes.
+#### remove-from-result-file  [Test Result]
+This option is used if there is a set of test results you wish to remove/delete from a saved results file. The user must specify a saved results file and then they will be prompted to select the results identifier associated with the results they wish to remove.
+#### remove-result  [Test Result]
+This option will permanently remove the saved file set that is set as the first argument.
+#### rename-identifier-in-result-file  [Test Result]
+This option is used if you wish to change the name of the identifier in a test results file that is shown in the Phoronix Test Suite Results Viewer and the contained graphs.
+#### rename-result-file  [Test Result]
+This option is used if you wish to change the name of the saved name of a result file.
+#### reorder-result-file  [Test Result]
+This option is used if you wish to manually change the order in which test results are shown in the Phoronix Test Suite Results Viewer and the contained graphs. The user must specify a saved results file and then they will be prompted to select the results identifiers one at a time in the order they would like them to be displayed from left to right.
+#### result-file-to-csv  [Test Result]
+This option will read a saved test results file and output the system hardware and software information along with the results to a CSV output. The CSV (Comma Separated Values) output can then be loaded into a spreadsheet for easy viewing.
+#### result-file-to-json  [Test Result]
+This option will read a saved test results file and output the basic result information to JSON (JavaScript Object Notation).
+#### result-file-to-pdf  [Test Result]
+This option will read a saved test results file and output the system hardware and software information along with the results to a PDF file.
+#### result-file-to-text  [Test Result]
+This option will read a saved test results file and output the system hardware and software information to the terminal. The test results are also outputted.
+#### show-result  [Test Result]
+Open up the test results in the Phoronix Test Suite Result Viewer or on OpenBenchmarking.org.
+#### winners-and-losers  [Test Result]
+This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.
+
+## Other
+#### commands
+This option will display a short list of possible Phoronix Test Suite commands.
+#### debug-dependency-handler
+This option is used for testing the distribution-specific dependency handler for external dependencies.
+#### debug-render-test
+This option is used during the development of the Phoronix Test Suite software for testing of the result and graph rendering code-paths This option will download a large number of reference test results from LinuxBenchmarking.com.
+#### debug-self-test
+This option is used during the development of the Phoronix Test Suite software for testing of internal interfaces, commands, and other common code-paths. The produced numbers should only be comparable for the same version of the Phoronix Test Suite, on the same hardware/software system, conducted on the same day of testing. This isn't intended as any scientific benchmark but simply to stress common PHP code-paths and looking for hot areas to optimize, etc.
+#### help
+This option will display a list of available Phoronix Test Suite commands and possible parameter types.
+#### version
+This option will display the Phoronix Test Suite client version.
+
+## Modules
+#### auto-load-module
+This option can be used for easily adding a module to the LoadModules list in the Phoronix Test Suite user configuration file. That list controls what PTS modules are automatically loaded on start-up of the Phoronix Test Suite.
+#### list-modules
+This option will list all of the available Phoronix Test Suite modules on this system.
+#### module-info  [Phoronix Test Suite Module]
+This option will show detailed information on a Phoronix Test Suite module such as the version, developer, and a description of its purpose.
+#### module-setup  [Phoronix Test Suite Module]
+This option will allow you to configure all available end-user options for a Phoronix Test Suite module. These options are then stored within the user's configuration file. Not all modules may have options that can be configured by the end-user.
+#### test-module  [Phoronix Test Suite Module]
+This option can be used for debugging a Phoronix Test Suite module.
+#### unload-module
+This option can be used for easily removing a module from the LoadModules list in the Phoronix Test Suite user configuration file. That list controls what modules are automatically loaded on start-up of the Phoronix Test Suite.
+
+## User Configuration
+#### enterprise-setup
+This option can be run by enterprise users immediately after package installation or as part of an in-house setup script. Running this command will ensure the phoronix-test-suite program is never interrupted on new runs to accept user agreement changes and defaults the anonymous usage reporting to being disabled and other conservative defaults.
+#### network-info
+This option will print information detected by the Phoronix Test Suite around the system's network configuration.
+#### network-setup
+This option allows the user to configure how the Phoronix Test Suite connects to OpenBenchmarking.org and other web-services. Connecting through an HTTP proxy can be configured through this option.
+#### user-config-reset
+This option can be used for resetting the Phoronix Test Suite user configuration file to its default state.
+#### user-config-set
+This option can be used for setting an XML value in the Phoronix Test Suite user configuration file.
+
+## Web / GUI Support
+#### gui
+Launch the Phoronix Test Suite HTML5 web user-interface in the local GUI mode (no remote web support) and attempt to auto-launch the web-browser. THIS FEATURE IS CURRENTLY EXPERIMENTAL AND NO LONGER ACTIVE DEVELOPMENT. See Phoronix Test Suite Phoromatic as an alternative web UI approach.
+
+## Phoromatic
+#### start-phoromatic-server
+Start the Phoromatic web server for controlling local Phoronix Test Suite client systems to facilitate automated and repeated test orchestration and other automated features targeted at the enterprise.
+
+# Module Options
+The following list is the modules included with the Phoronix Test Suite that are intended to extend the functionality of pts-core. Some of these options have commands that can be run directly in a similiar manner to the other Phoronix Test Suite user commands. Some modules are just meant to be loaded directly by adding the module name to the LoadModules tag in ~/.phoronix-test-suite/user-config.xml or via the PTS_MODULES environment variable. A list of available modules is also available by running *phoronix-test-suite list-modules.*
+
+---
+
+### Backup Creation + Restore
+This is a module for creating backups of the Phoronix Test Suite / Phoromatic and allows for restoring of created backups. The backup will be in ZIP or TAR format. If only a path is specified, the file-name will be auto-generated with a current time-stamp.
+phoronix-test-suite backup.create
+phoronix-test-suite backup.restore
+
+### Dummy Module
+This is a simple module intended for developers to just demonstrate some of the module functions.
+phoronix-test-suite dummy_module.dummy-command
+
+### Generate Perf FlameGraphs For Tests
+Setting FLAME_GRAPH_PATH=<path to flamegraph path> will auto-load and enable this Phoronix Test Suite module. The module will generate a Linux perf FlameGraph for each test run during the benchmarking process. Details on FlameGraph @ https://github.com/brendangregg/FlameGraph
+This module utilizes the following environmental variables: FLAME_GRAPH_PATH.
+
+### Graphics Override
+This module allows you to override some graphics rendering settings for the ATI and NVIDIA drivers while running the Phoronix Test Suite.
+This module utilizes the following environmental variables: FORCE_AA, FORCE_AF.
+
+### Result Exporter To HTML
+This module allows basic exporting of results to HTML for saving either to a file locally (specified using the EXPORT_RESULTS_HTML_FILE_TO environment variable) or to a mail account (specified using the EXPORT_RESULTS_HTML_EMAIL_TO environment variable). EXPORT_RESULTS_HTML_EMAIL_TO supports multiple email addresses delimited by a comma.
+This module utilizes the following environmental variables: EXPORT_RESULTS_HTML_EMAIL_TO, EXPORT_RESULTS_HTML_FILE_TO.
+
+### Linux Perf Framework Reporter
+Setting LINUX_PERF=1 will auto-load and enable this Phoronix Test Suite module. The module also depends upon running a modern Linux kernel (supporting perf) and that the perf binary is available via standard system paths.
+This module utilizes the following environmental variables: LINUX_PERF.
+
+### Log Exporter
+This module allows for easily exporting test run logs and system logs to external locations via specifying the directory paths via the COPY_TEST_RUN_LOGS_TO and COPY_SYSTEM_LOGS_TO environment variables.
+This module utilizes the following environmental variables: COPY_TEST_RUN_LOGS_TO, COPY_SYSTEM_LOGS_TO.
+
+### MATISK
+My Automated Test Infrastructure Setup Kit
+phoronix-test-suite matisk.run
+phoronix-test-suite matisk.template
+
+### OpenBenchmarking.org Auto Comparison
+This module prints comparable OpenBenchmarking.org results in the command-line for reference purposes as tests are being run. OpenBenchmarking.org is automatically queried for results to show based on the test comparison hash and the system type (mobile, desktop, server, cloud, workstation, etc). No other system information or result data is transmitted..
+phoronix-test-suite ob_auto_compare.debug
+
+### Performance Per Dollar/Cost Calculator
+Setting the COST_PERF_PER_DOLLAR= environment variable to whatever value of the system cost/component you are running a comparison on will yield extra graphs that calculate the performance-per-dollar based on the test being run. The COST_PERF_PER_DOLLAR environment variable is applied just to the current test run identifier. Set the COST_PERF_PER_UNIT= environment variable if wishing to use a metric besides dollar/cost. The COST_PERF_PER_HOUR value can be used rather than COST_PERF_PER_DOLLAR if wishing to calculate the e.g. cloud time or other compute time based on an hourly basis.
+phoronix-test-suite perf_per_dollar.add
+This module utilizes the following environmental variables: COST_PERF_PER_DOLLAR, COST_PERF_PER_UNIT, COST_PERF_PER_HOUR.
+
+### Performance Tip Prompts
+This module alerts the user if the system configuration may not be the right one for achieving the best performance with the target benchmark(s). This initial version of the module actually cares only about the BFQ I/O scheduler and powersave governor checks.
+phoronix-test-suite perf_tips.show
+This module utilizes the following environmental variables: SUPPRESS_PERF_TIPS.
+
+### Benchmarking Compiler PGO Impact
+This module makes it easy to test a compiler PGO (Profile Guided Optimization) performance impact by running a test without PGO optimizations, capturing the PGO profile, rebuilding the tests with the PGO profile generated, and then repeat the benchmarks.
+phoronix-test-suite pgo.benchmark
+
+### Phoromatic Client
+The Phoromatic client is used for connecting to a Phoromatic server (Phoromatic.com or a locally run server) to facilitate the automatic running of tests, generally across multiple test nodes in a routine manner. For more details visit http://www.phoromatic.com/. This module is intended to be used with Phoronix Test Suite 5.2+ clients and servers.
+phoronix-test-suite phoromatic.connect
+phoronix-test-suite phoromatic.explore
+phoronix-test-suite phoromatic.upload-result
+phoronix-test-suite phoromatic.set-root-admin-password
+phoronix-test-suite phoromatic.list-results
+phoronix-test-suite phoromatic.clone
+phoronix-test-suite phoromatic.export-results-for-account-schedules
+
+### Pushover.net
+Submit notifications to your iOS/Android mobile devices of test results in real-time as push notifications, etc. Using the Pushover.net API.
+This module utilizes the following environmental variables: PUSHOVER_NET_USER.
+
+### Report Test Time Graphs
+Setting the RUN_TIMES_ARE_A_BENCHMARK=1 environment variable will automatically create additional graphs for each test run plotting the run-time needed for each test being executed. Setting the INSTALL_TIMES_ARE_A_BENCHMARK=1 environment variable will automatically create additional graphs for each test run plotting the time required for the test installation. Setting the INSTALL_SIZES_ARE_A_BENCHMARK=1 environment variable will automatically create additional graphs for each test run plotting the size of the installed test directory.
+This module utilizes the following environmental variables: RUN_TIMES_ARE_A_BENCHMARK, INSTALL_TIMES_ARE_A_BENCHMARK, INSTALL_SIZES_ARE_A_BENCHMARK.
+
+### Result Notifier
+A notification module.
+
+### Custom Result Export Methods
+A simple example module about interfacing with Phoronix Test Suite core for dumping result files in a custom format.
+phoronix-test-suite results_custom_export.nf
+
+### System Monitor
+This module contains sensor monitoring support.
+This module utilizes the following environmental variables: MONITOR, PERFORMANCE_PER_WATT, MONITOR_INTERVAL, MONITOR_PER_RUN.
+
+### Timed Screenshot
+This is a module that will take a screenshot of the system at a pre-defined interval. ImageMagick must be installed onto the system prior to using this module.
+This module utilizes the following environmental variables: SCREENSHOT_INTERVAL.
+
+### Toggle Screensaver
+This module toggles the system's screensaver while the Phoronix Test Suite is running. At this time, the GNOME and KDE screensavers are supported.
+This module utilizes the following environmental variables: HALT_SCREENSAVER.
+
+### Update Checker
+This module checks to see if the Phoronix Test Suite -- and its tests and suites -- are up to date.
+
+### Utilize Wine On Linux Benchmarking
+This module when activated via the USE_WINE environment variable on Linux systems will override the test profile OS target to Windows and attempt to run the (Windows) tests under Wine, if installed on the system. USE_WINE can be either set to the name of the desired wine command or the absolute path to the wine binary you wish to use for benchmarking.
+This module utilizes the following environmental variables: USE_WINE.
+
+### System Event Watchdog
+This module has support for stopping/interrupting tests if various system issues occur, like a temperature sensor exceeds a defined threshold.
+This module utilizes the following environmental variables: WATCHDOG_SENSOR, WATCHDOG_SENSOR_THRESHOLD.
+
+# Installation Instructions
+
+## Setup Overview
+The Phoronix Test Suite supports Linux, Apple Mac OS X, Microsoft Windows, Solaris, Hurd, BSD, and other operating system environments. The only Linux distribution-specific code deals with the external dependencies support feature that are set by individual test profiles. If you are not running one of the supported Linux distributions, Solaris, BSD, or Mac OS X, you may need to install a package manually (as instructed by the Phoronix Test Suite) in order for a test to run. An example of an external dependency would be GCC and the OpenGL Utility Toolkit being needed for test profiles that build an OpenGL benchmark from source-code.
+Among the distributions where the Phoronix Test Suite has been officially tested include Ubuntu, Fedora, Mandriva / Mageia, Gentoo, PCLinuxOS, Arch Linux, Pardus, OpenSuSE, Optware, webOS, Zenwalk, CentOS, Red Hat Enterprise Linux, Oracle Linux, Scientific Linux, Debian, Mint, MEPIS, Alpine Linux, Void Linux, Intel Clear Linux, and Amazon Linux EC2.
+Among the tested BSD distributions are FreeBSD, PC-BSD, NetBSD, OpenBSD, Debian GNU/kFreeBSD, and DragonflyBSD. Tested Solaris distributions include OpenSolaris, Solaris Express 11, Oracle Solaris 11, OpenIndiana, Illumos, and Nexenta.
+
+## Dependencies
+The only required dependency for the Phoronix Test Suite is PHP 5.3 or newer. On Linux distributions, the needed package is commonly called *php5-cli* or *php-cli* or *php7* or *php* . It is important to note that only PHP for the command-line is needed and not a web server (Apache) or other packages commonly associated with PHP and its usage by web-sites. The PHP5 version required is PHP 5.3+ and can also be found at [www.php.net](http://www.php.net/) . PHP 7 is fully supported by the Phoronix Test Suite as well as HHVM.
+For installing PHP on Windows, the [Microsoft Web Platform Installer](https://www.microsoft.com/web/platform/phponwindows.aspx) provides an automated alternative that will install PHP into the directory expected by Phoronix Test Suite. Users opting to install PHP manually must extract the files to *C:\Program Files (x86)\PHP\* or a subdirectory.
+As part of the PHP requirement, the following PHP extensions are required and/or highly recommended in order to take advantage of the Phoronix Test Suite capabilities:
+
+
++
++ PHP DOM is needed for XML operations and must be installed for the Phoronix Test Suite to function.
++
++ PHP ZIP is needed for file compression and decompression and specifically dealing with test profiles and suites obtained via OpenBenchmarking.org or when uploading such tests and suites.
++
++ PHP OpenSSL is used for enabling HTTPS communication with Phoronix Test Suite / OpenBenchmarking.org servers.
++
++ PHP GD is highly recommended for rendering of test data to JPEG and PNG image formats and is selectively used in other image operations.
++
++ PHP Zlib is highly recommended for greater data compression when dealing with remote OpenBenchmarking.org assets.
++
++ PHP PCNTL is used for multi-threaded system sensor monitoring support during the testing process and other threaded tasks by the Phoronix Test Suite module framework.
++
++ PHP POSIX is used for reliably obtaining more system information in an efficient manner.
++
++ PHP CURL is supported as an alternative networking library for improved network performance in downloading test files and other operations.
++
++ PHP FPDF is used to generate PDF reports of test data.
++
+Without all of these extensions, some capabilities of the Phoronix Test Suite will not be available. Many of these packages are enabled by default and do not require any additional installation steps on most Linux distributions, otherwise they are often found in the package vendor's repository.
+
+## Notes
+
+### General
+You may need to modify the *php.ini* file on your system in order to support uploading results to OpenBenchmarking.org or logging into your OpenBenchmarking.org account. The *allow_url_fopen* , *file_uploads* , and *allow_url_include* options must be set to true in the PHP configuration. See additional information in this [Phoronix Forums support thread](http://www.phoronix.com/forums/showpost.php?p=32484&postcount=4) .
+Major updates to the Phoronix Test Suite are released on a quarterly basis. The latest stable and development versions of the Phoronix Test Suite are available at [Phoronix-Test-Suite.com](http://www.phoronix-test-suite.com/) . The Git repository where the latest Phoronix Test Suite code is provided is hosted at [GitHub.com/phoronix-test-suite](https://github.com/phoronix-test-suite/phoronix-test-suite) and can be cloned/pulled from the *https://github.com/phoronix-test-suite/phoronix-test-suite.git* repository location. The latest upstream development code is housed in the master tree while older Phoronix Test Suite releases are available in their respective Git branches based upon the release's code-name.
+If building the PHP package from upstream sources, it should just be a matter of running *./configure* with the *--enable-zip* flag (all other requirements should be apart of the stock PHP configuration) to satisfy the PHP needs of the Phoronix Test Suite.
+
+### File Structure
+If manually changing the location of the *phoronix-test-suite* launcher file, the *PTS_USER_PATH* environment variable must be adjusted inside the file to reflect the absolute location that leads to the root directory of the *pts* and *pts-core* folders. The *pts-core* directory contains the "engine" of the Phoronix Test Suite.
+
+### Running Locally
+The Phoronix Test Suite can be simply extracted from the downloaded *.tar.gz* or *.zip* file or it can also be installed system-wide. If you just wish to run the Phoronix Test Suite without installing it, open a terminal and run *./phoronix-test-suite <options>* from the same directory.
+
+### Generic Installation
+Running *install-sh* from the root directory of the Phoronix Test Suite will install the software for system-wide access. By default the *phoronix-test-suite* executable is in */usr/bin/* , the Phoronix Test Suite files in */usr/share/phoronix-test-suite/* , and the documentation in / *usr/share/doc/phoronix-test-suite/* . Root access is required. The default installation prefix is /usr/ but can be adjusted as the first argument (example: *install-sh /home/user/* to install the Phoronix Test Suite in your home directory).
+
+### Debian/Ubuntu Installation
+Debian/Ubuntu users are able to follow the Generic Installation instructions or can obtain a Debian Package from the Phoronix Test Suite web-site. The package contains the *phoronix-test-suite* executable in */usr/bin/* , the Phoronix Test Suite files in */usr/share/phoronix-test-suite/* , and the documentation in */usr/share/doc/phoronix-test-suite/* .
+A phoronix-test-suite package can also be found in the official Ubuntu universe repository beginning with the Ubuntu 9.04 (Jaunty) release, but it may be out of date compared to the latest upstream version.
+On Ubuntu 13.04 and newer if you experience an issue of system sensor monitoring not working, other multi-process tasks, or related issues reported as PHP PCNTL missing, the pcntl functions need to be removed from the *disable_functions* line within the */etc/php5/cli/php.ini* file. This may also affect other Linux distribution support too if the vendor disables access to the PCNTL functions.
+
+### Fedora / Red Hat Installation
+The Phoronix Test Suite can be installed on Fedora, Red Hat Enterprise Linux, and CentOS systems using the generic installation method. Alternatively, a *phoronix-test-suite* package is available in recent versions of the Fedora repository and in the EPEL (Extra Packages for Enterprise Linux) repository for Red Hat Enterprise Linux. At this time, however, the Fedora-supplied package is vastly out of date.
+
+### Gentoo Installation
+Gentoo ebuild's for the Phoronix Test Suite have been created by users and can be currently found in the [Gentoo BugZilla](http://bugs.gentoo.org/show_bug.cgi?id=216656) .
+
+### Mandriva Installation
+Phoronix Test Suite packages are available through Mandriva Cooker starting with the Mandriva Linux 2009 release.
+
+### OpenSolaris Installation
+The Phoronix Test Suite core (pts-core) is supported on Solaris and OpenSolaris platforms, but some features may not fully function. Additionally, the PTS External Dependencies feature is supported on OpenSolaris using IPS (the Image Packaging System). However, not all test profiles are supported on Solaris / OpenSolaris, and it may be in violation of your end-user agreement to perform such benchmarks on the Sun Solaris operating system. Using the OpenSolaris Image Packaging System, installing *SUNWphp524* will provide PHP, PHP GD, and related dependencies. The Phoronix Test Suite has also been tested on Oracle Solaris 11 Express, OpenIndiana, StormOS, and Nexenta Core Platform.
+
+### BSD Installation
+The Phoronix Test Suite also supports *BSD operating systems. However, like the Solaris support, not all test profiles are compatible with BSD operating systems, but pts-core has been tested against FreeBSD 7.x/8.x/9.x, NetBSD, PC-BSD 7/8/9, DragonFlyBSD 3.0/3.2/3.4/3.6/4.2, and OpenBSD 4.4/4.6. The Phoronix Test Suite will also work with Debian GNU/kFreeBSD.
+For DragonFlyBSD, it's a matter of just using *pkg_radd* with php5, php5-json, php5-dom, and php5-zip or *pkgsrc* . With DragonFlyBSD using dports, the command is *pkg install php5 php5-dom php5-zip php5-json php5-simplexml* . Under FreeBSD/PC-BSD, PHP can be obtained from the *php5* and *php5-extensions* Ports collection.
+
+### Mac OS X Installation
+The Phoronix Test Suite is fully supported on Apple's Mac OS X operating system as of Mac OS X 10.5 with improved support under Mac OS X 10.6. PHP ships with Mac OS X by default so it's simply a matter of downloading the Phoronix Test Suite package, extracting it, and running the executable. For tests that rely upon a compiler, Apple's XCode with GCC and LLVM can be utilized.
+
+### Windows Installation
+The Phoronix Test Suite support on Microsoft Windows is in development and will function just fine, but is not yet feature complete compared to the other operating systems support. Only a small subset of test profiles, mostly related to graphics, are supported. In terms of the version support, right now the Phoronix Test Suite is developed and tested solely against Microsoft Windows 7 x64 and Microsoft Windows 8 x64.
+At this time the Phoronix Test Suite client is dependent upon the *php.exe* executable being installed to *C:\Program Files (x86)\PHP\ * or one of its subdirectories (see PHP installation notes above). Once this requirement is met, simply download the Phoronix Test Suite package, extract it to a directory of your choice, and run *phoronix-test-suite.bat*
+NOTE: because of lack of external dependencies support, it is necessary to manually download and install the files for some test profiles, such as the Unigine graphics benchmarks. See Troubleshooting for more information.
+
+### Facebook HipHop
+The Phoronix Test Suite can use Facebook's [HipHop HHVM](https://github.com/facebook/hiphop-php) virtual machine as an alternative to the de facto PHP implementation. The Phoronix Test Suite has been tested against HHVM 2.0. If HHVM is present on the system but not PHP, it will automatically be used by the Phoronix Test Suite. Alternatively, the *PHP_BIN* environment variable can be set to point to an alternative *hhvm* binary.
+The Phoronix Test Suite also supports the older version of Facebook HipHop that serves as a PHP to C++ translator and compiler with GCC. This support though is primitive and not officially supported. To compile the Phoronix Test Suite using HipHop/GCC, run *find . -name "*.php" > files.list && hphp --input-list=files.list -k 1 --log=3 --include-path="." --cluster-count=50 -v "AllDynamic=true" -v "AllVolatile=true"* from the root *phoronix-test-suite/* directory. It can then be executed in the form of */tmp/hphp_XXX/program -f pts-core/phoronix-test-suite.php system-info* .
+
+# External Dependencies
+The Phoronix Test Suite has a feature known as "External Dependencies" where the Phoronix Test Suite can attempt to automatically install some of the test-specific dependencies on supported distributions. If running on a distribution where there is currently no External Dependencies profile, the needed package name(s) are listed for manual installation.
+Below are a list of the operating systems that currently have external dependencies support within the Phoronix Test Suite for the automatic installation of needed test files.
+
+---
+
+
+Alpine LinuxAmazonAngstromArch LinuxCentOSClearOSClearOS Core ServerDebianDragonFlyBSDFedoraFluxbuntuGNU KFreeBSDGentooGoobuntuHPJoli CloudLinaroLinux Embedded Development EnvironmentLinux MintMEPISMac OS XMacPortsMageiaMandrivaMeeGoMicrosoft WindowsMidnightBSDMoblinMythbuntuNetBSDNexenta CoreOLPCOpenIndianaOpenMandrivaOpenSolarisOpenSuSEOptwareOracle ServerPCLinuxOSPTS Desktop LivePTS Linux LivePalmPardus LinuxRed Hat EnterpriseRed Hat Enterprise ServerSUSESUSE LinuxScientificScientificSLSolusSolus LinuxTermuxUbuntuVoid LinuxZenwalkgNewSensemacOS Brew
+# Configuration
+
+## User Files & Folders
+**~/.phoronix-test-suite/user-config.xml**
+This is a per-user configuration file. Among the information stored here is the test options, locations for storing files, and batch mode options. This file is formatted in XML.
+**~/.phoronix-test-suite/graph-config.json**
+This is a per-user configuration file for storing graph attributes. The adjustable options include HTML hex color codes for different areas of the graph, dimensions of the graph, and font sizes. This file is formatted in JSON.
+**~/.phoronix-test-suite/download-cache/**
+This directory contains test packages that have been downloaded for test profiles. For more information on the download cache.
+**~/.phoronix-test-suite/installed-tests/**
+This directory is where tests are installed by default. Each test has its own directory within a sub-directory of *installed-tests/* based upon its OpenBenchmarking.org repository. In the test's folder is a *pts-install.xml* file used for managing the installation.
+**~/.phoronix-test-suite/test-results/**
+This directory is where tests results are saved by default. Each saved file has its own directory. In the saved directory is then a *composite.xml* file containing the useful results while in the *test-X.xml* files are back-ups of the results.
+**~/.phoronix-test-suite/modules-data/**
+This is the directory where any Phoronix Test Suite modules should save any files to, within a sub-directory of the module's name. The module configuration settings are also stored within this directory.
+**~/.phoronix-test-suite/test-profiles/**
+This is the directory where test profiles are stored.
+**~/.phoronix-test-suite/test-suites/**
+This is the directory where test suites are stored.
+
+## Environment Variables
+**TEST_RESULTS_NAME**
+When this variable is set, the value will be used as the name for automatically saving the test results.
+**TEST_RESULTS_IDENTIFIER**
+When this variable is set, the value will be used as the test identifier when automatically saving the test results.
+**TEST_RESULTS_DESCRIPTION**
+When this variable is set, the value will be used as the test results description when saving the test results.
+**PRESET_OPTIONS**
+For setting any test option(s) from an environment variable rather than being prompted for the options when running a test. Example: *PRESET_OPTIONS="stream.run-type=Add" ./phoronix-test-suite benchmark stream* . Multiple options can be passed to this environment variable when delimited by a semicolon.
+**SKIP_TESTS**
+If there are any test(s) to exempt from the testing process, specify them in this variable. Multiple tests can be waived by delimiting each test identifier by a comma. A test hardware type (i.e. Graphics) can also be supplied for skipping a range of tests.
+**SKIP_TESTS_HAVING_ARGS**
+If any of the test(s) have an argument matching any strings contained in this environment variable, the test execution will be skipped. Multiple strings can be set when delimiting by a comma.
+**RUN_TESTS_IN_RANDOM_ORDER**
+Setting this environment variable will cause the tests to be run in a random order.
+**SKIP_TESTING_SUBSYSTEMS**
+If you are running a set of benchmarks (namely a result file) but wish to skip some of the tests that don't belong to a certain test type group, you can set the hardware types to test via this environment variable. E.g. setting *SKIP_TESTING_SUBSYSTEMS=Graphics* will skip all test profiles to run that are not of the graphics test group. Multiple types should be delimited by a comma.
+**PTS_MODULE_SETUP**
+This variable can be used to load Phoronix Test Suite module settings automatically when using the *module-setup* option. An example would be: *PTS_MODULE_SETUP="phoromatic.remote_host=http://www.phoromatic.com/; phoromatic.remote_account=123456; phoromatic.remote_verifier=ABCD" phoronix-test-suite module-setup phoromatic* .
+**PTS_MODULES**
+If there are any Phoronix Test Suite modules to additionally load, they can be specified here. Multiple modules can be supplied by delimiting them with a comma. The more appropriate way of loading Phoronix Test Suite modules for longer periods of time is by using the *~/.phoronix-test-suite/user-config.xml* configuration.
+**NO_PHODEVI_CACHE**
+This is a debugging option to disable the Phodevi cache from being loaded of cached software/hardware information. Instead, all software/hardware will be polled from the Phodevi library without caching.
+**EXTERNAL_PHODEVI_CACHE**
+This option can be used for loading an external Phodevi cache. Such as loading the native hardware/software information from within a Windows Wine client from a native system host.
+**PTS_DISPLAY_MODE**
+If you wish to load a non-default display mode for a single instance, specify the mode in this variable.
+**TOTAL_LOOP_TIME**
+When running any test(s), if you would like the test(s) to continue running as a loop until a certain time has been reached, this variable can be used. The value should be the number of minutes to run the testing process before the loop is ended. The testing will finish whenever the currently active test has finished once the time has elapsed. The minimum value allowed is 10 minutes.
+**LIMIT_ELAPSED_TEST_TIME**
+If you want to ensure that the time for a given Phoronix Test Suite process doesn't elapse past a certain number of minutes, specify the number of minutes for this environment variable. When the amount of time spent testing exceeds that amount, the testing will end prematurely while still saving the tests that were completed in time.
+**TOTAL_LOOP_COUNT**
+When running any test(s), if you would like the test(s) to continue running for a number of times, this variable can be used. The value should be the number of times to loop the testing process before ending.
+**FORCE_TIMES_TO_RUN**
+If you wish to override the number of times to run each test -- rather than the Phoronix Test Suite using the number of times specified in each test profile -- this variable can be used.
+**FORCE_MIN_TIMES_TO_RUN**
+This is similar to the FORCE_TIMES_TO_RUN option but will only be used if the test profile's run count is less than this defined value.
+**NO_FILE_HASH_CHECKS**
+To disable MD5/SHA256 check-sums from being checked when downloading test files, set this variable to 1. This variable used to be known as *NO_MD5_CHECKS* , which is still honored but was changed to *NO_FILE_HASH_CHECKS* to reflect other kind of file hash sum checks.
+**NO_HTTPS**
+Set this environment variable to 1 if you don't wish to use HTTPS download links for test profiles (or the system/network lacks HTTPS support). When enabled, HTTPS links will then be done over HTTP.
+**PTS_DOWNLOAD_CACHE**
+While non-standard Phoronix Test Suite download caches can be specified within the *user-config.xml* file, an additional directory to look for potential Phoronix Test Suite download files can be specified by this variable.
+**GRAPH_HIGHLIGHT**
+If this variable is set with a valid test identifer from a result file whether you are using the *refresh-graphs* command or any other related to the rendering of test results on a bar graph, the specified test identifier's result will be rendered in a different color than the other test results. Multiple identifiers can be specified when delimited by a comma. Additionally, for each key it is possible to provide the actual color value, or an index in the color palette. Example: "will_be_different,group1a=1,group1b=1,blue=#0000ff"
+**TEST_EXEC_PREPEND**
+Set this variable to any command/environment variable that you may be passed prepended to the test execution string at runtime.
+**VIDEO_MEMORY**
+If Phodevi fails to detect the system's video memory capacity or is incorrectly detected, the video memory capacity (in MB) can be specified by this variable.
+**OVERRIDE_VIDEO_MODES**
+If Phodevi fails to detect all of the system's monitor video modes or a separate set of modes would be preferred, the modes can be specified in this variable. Example: *OVERRIDE_VIDEO_MODES=800x600,1024x768,1280x1024 phoronix-test-suite benchmark nexuiz* .
+**SKIP_TEST_SUPPORT_CHECKS**
+If this environment variable is set, it will not honor the support checks made by individual test profiles. I.e. test profiles that would normally be considered un-supported on a given platform are attempted to install and run regardless.
+**SKIP_ALL_TEST_SUPPORT_CHECKS**
+If this environment variable is set, all tests will be permitted on the client for execution. SKIP_ALL_TEST_SUPPORT_CHECKS is more liberal than SKIP_TEST_SUPPORT_CHECKS in letting disk tests run on RAM-based file-systems, attempt to run 2D/3D tests on VESA display drivers, and other special cases.
+**DEFAULT_VIDEO_MODE**
+If Phodevi fails to detect the system's monitor standard / default resolution, the mode can be specified in this variable. Example: *DEFAULT_VIDEO_MODE=1680x1050 phoronix-test-suite benchmark nexuiz* .
+**SKIP_EXTERNAL_DEPENDENCIES**
+To skip the Phoronix Test Suite external dependency checking/installation when installing a test, set this environment variable to *1* . If wishing to skip only certain external dependencies, set this variable's value to the name of the external dependencies (the generic dependency names used by the Phoronix Test Suite) to not install. Multiple dependencies to skip can be delimited by a comma.
+
+# General Information
+
+## Frequently Asked Questions
+**Q: May I use the Phoronix Test Suite when running benchmarks for my own publication or blog? Are there any publishing restrictions?**
+**A:** Anyone is more than welcome to use the Phoronix Test Suite for their own publication or purpose. While the Phoronix Test Suite came out of our internal test tools for carrying out Linux hardware reviews at [Phoronix.com](http://www.phoronix.com/) , we invite other hardware review web-sites, technology journals, and independent publications to use our software too. While not required, we would just kindly ask that you mention in your review/article that the *Phoronix Test Suite* was used for carrying out your testing, and ideally to link to [www.phoronix-test-suite.com](http://www.phoronix-test-suite.com/) so that your readers will know where to obtain the software if they are interested in running the tests. You are also more than welcome to upload your results to [OpenBenchmarking.org](http://www.openbenchmarking.org/) so that others may compare their results against yours in an easy manner.
+We also try to make the Phoronix Test Suite easy-to-use by independent publications. For example, if you would like to watermark your web-site's URL into the graphs containing your test results, that can be easily modified in *~/.phoronix-test-suite/graph-config.json* . The colors and other graph settings are also stored in this XML file. If you are a publication and run into any issues with the Phoronix Test Suite or have a feature request, please let us know.
+A partial list of other publications using the Phoronix Test Suite can be found in [this forum thread](http://www.phoronix.com/forums/showthread.php?t=12925) .
+**Q: Why does the Phoronix Test Suite not use my distribution's package management system for acquiring all needed packages?**
+**A:** The tests themselves are generally downloaded from source and built locally on the machine, rather than fetching any distribution-specific packages. This is done to ensure more comparable results across operating systems / releases, etc. The distribution packager could be applying a number of unknown patches to the software, building the software with unique build options, or making other changes to the software that could skew the results.
+**Q: Besides being a developer, documentation writer, or having any other unique technical abilities, how else can I contribute to the Phoronix Test Suite?**
+**A:** Independent code contributions are very welcome as well as creating your own test profiles and suites. We also appreciate any feedback, comments, or other ideas either by emailing us, posting on the forums, or sending a message to the mailing list. You can also see this (outdated) [forum thread](http://www.phoronix.com/forums/showthread.php?t=16180) .
+**Q: Do you offer technical support for the Phoronix Test Suite**
+**A:** Paid, professional support is available and is done via [our commercial services](http://commercial.phoronix-test-suite.com/) . We also offer Phoromatic licenses for use within a corporate intranet and other custom services. Free, community support is offered via our [mailing list](http://phoronix-test-suite.com/mailman/listinfo/trondheim-pts_phoronix-test-suite.com) , IRC channel ( *#phoronix* on *FreeNode.net* , and the [Phoronix Forums](http://www.phoronix.com/forums/) .
+**Q: May I put the Phoronix Test Suite logo on my company's web-site or on my product packaging?**
+**A:**  [Contact us](http://www.phoronix-test-suite.com/?k=contact) for licensing information and details regarding the Phoronix Certification & Qualification Suite.
+**Q: How often is the Phoronix Test Suite updated?**
+**A:** We provide major updates on a quarterly basis with an occasional point release to fix outstanding bugs or address other issues. The latest work going into the Phoronix Test Suite is accessible via our Git repository at [GitHub.com/phoronix-test-suite](https://github.com/phoronix-test-suite/phoronix-test-suite/) .
+
+## Tips & Tricks
+
+### General
+- The desktop's screensaver will automatically be shutdown when a test is running and will be restored to its previous state upon the test's completion. This is supported for GNOME, KDE, and other XDG-supportive desktop environments.
+- If you have many computers you wish to benchmark, once all of your tests have been downloaded, run *phoronix-test-suite make-download-cache* to generate a copy of these files at *~/.phoronix-test-suite/download-cache/* . A download cache is used for conserving time and bandwidth by eliminating the need for the Phoronix Test Suite to download files that have already been downloaded once. Copy this folder to the other systems or copy it to a DVD or USB hard drive, connect it to the next test system, and the Phoronix Test Suite will automatically use this local download cache. Or store these files on a local HTTP/FTP server and then update your *~/.phoronix-test-suite/user-config.xml* file to reflect the location of this download cache directory.
+
+### Running
+- When running a test in batch mode (through the use of the *batch-run* or *batch-benchmark* options) that normally has end-user options (such as the sub-test to run or resolution), the Phoronix Test Suite will run the test with each unique combination of options possible, if configured appropriately.
+- When running a test where you are prompted to enter any test options, multiple selections can be performed -- which will result in multiple test runs for each combination of the selected option(s) -- by separating each intended test option / number with a comma.
+- When being prompted for the test identifier or the file name for saving the results, several user variables are supported. These include *$VIDEO_RESOLUTION* , *$VIDEO_CARD* , *$OPERATING_SYSTEM* , *$PROCESSOR* , *$MOTHERBOARD* , *$CHIPSET* , and *$KERNEL_VERSION* . If any of these variables are entered, the Phoronix Test Suite will replace them with their respective values before saving the results.
+- If *RemoveDownloadFiles* is set to *TRUE* within the *user-config.xml* file, once a test has been installed the originally downloaded files for that test will be automatically removed. This conserves disk space but will cause these files to be re-downloaded the next time the test needs to be re-installed. This will also not back up the downloaded files to the Phoronix Test Suite download cache. Enabling this option is just recommended for users with very limited disk space.
+- If the amount of video memory for your graphics card is incorrectly reported by the Phoronix Test Suite (you can check by running *phoronix-test-suite diagnostics* ), you can use the *VIDEO_MEMORY=* environment variable for overriding the video memory capacity (in Megabytes) used by the Phoronix Test Suite.
+- If the *DISPLAY* environment variable is not set or *NO_GRAPHICS_TESTS* environment variable is set, tests of type *Graphics* will not be run. Likewise, if *NO_SYSTEM_TESTS* environment variable is set, tests of type *System* will not run. This applies to all test types where *NO_<TEST TYPE>_TESTS* is set.
+- If while running multiple tests you want to quit the testing prematurely, in a new terminal window type *touch ~/.phoronix-test-suite/halt-testing* . All results for tests that had already run will be saved (permitting you opted to save the results), except for the test currently being run.
+- If you wish to stop the current test run prematurely but continue the testing process, in a new terminal window type *touch ~/.phoronix-test-suite/skip-test* .
+- If you want the specified test(s) to run in a loop for a set period of time, use the *TOTAL_LOOP_TIME* environment variable. For instance, running *TOTAL_LOOP_TIME=120 phoronix-test-suite benchmark ffmpeg* would keep running the ffmpeg test profile for 120 minutes.
+- If you want the specified test(s) to run in a loop for a set number of times, use the *TOTAL_LOOP_COUNT* environment variable. For instance, running *TOTAL_LOOP_COUNT=3 phoronix-test-suite benchmark ffmpeg* would keep running the ffmpeg test profile three times.
+- When any tests are being installed and when tests are being run, a lock is created in the system's temporary directory with the name *phoronix-test-suite.active* (i.e. */tmp/phoronix-test-suite.active* ) and is removed upon completion. Thus if you have any system scripts that you wish to run when tests are not running or being installed as to not impact the results, one simple way to handle this is by having the script check for the existence of this lock.
+
+### Troubleshooting
+In the event that a test profile fails to install or run, the following general troubleshooting steps may be helpful:- If a test profile fails immediately after starting, check the test profile's directory in *~/.phoronix-test-suite/installed-tests/* to confirm that the needed files are present. On platforms without External Dependencies support (Windows), it may be necessary to download the files manually and place them in this directory. If this is the case, you will notice that the "Downloading" phase of test installation completes instantly.
+- Inspect the scripts inside the above test profile's directory and confirm that directories or search paths for the test correspond to those on your system
+- Try running the test profile with the *debug-benchmark* command, or reinstalling with the *debug-install* command and make note of any unusual output.
+
+### Configuration
+- The user configuration options for the Phoronix Test Suite are stored in *~/.phoronix-test-suite/user-config.xml* . The batch mode options are also stored within this file and those can be adjusted by running *phoronix-test-suite batch-setup* .
+- The colors, size, and other attributes for the graphs found within the Phoronix Test Suite Results Viewer can be modified via the file *~/.phoronix-test-suite/graph-config.json* .
+
+### Test / Suite Writing
+- The Phoronix Test Suite recursively determines tests/suites and allows a suite to call another suite.
+
+### Module Writing
+- By writing a Phoronix Test Suite module, you can easily extend the functionality of the Phoronix Test Suite. Some examples are being able to email the results upon completion, controlling the system's screensaver, updating a text LCD panel with the current test status, etc.
+
+# Virtual Test Suites
+Virtual test suites are not like a traditional test suite defined by the XML suite specification. Virtual test suites are dynamically generated in real-time by the Phoronix Test Suite client based upon the specified test critera. Virtual test suites can automatically consist of all test profiles that are compatible with a particular operating system or test profiles that meet other critera. When running a virtual suite, the OpenBenchmarking.org repository of the test profiles to use for generating the dynamic suite must be prefixed.
+Virtual test suites can be installed and run just like a normal XML test suite and shares nearly all of the same capabilities. However, when running a virtual suite, the user will be prompted to input any user-configuration options for needed test profiles just as they would need to do if running the test individually. When running a virtual suite, the user also has the ability to select individual tests within the suite to run or to run all of the contained test profiles. Virtual test suites are also only supported for an OpenBenchmarking.org repository if there is no test profile or test suite of the same name in the repository. Below is a list of common virtual test suites for the main Phoronix Test Suite repository, but the dynamic list of available virtual test suites based upon the enabled repositories is available by running *phoronix-test-suite list-available-virtual-suites.*
+
+---
+#### All PTS Tests  pts/all
+This is a collection of all test profiles found within the specified OpenBenchmarking.org repository.
+#### Installed Tests  pts/installed
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository that are already installed on the system under test.
+#### Linux Operating System Tests  pts/linux
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Linux Operating System.
+#### Solaris Operating System Tests  pts/solaris
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Solaris Operating System.
+#### BSD Operating System Tests  pts/bsd
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the BSD Operating System.
+#### MacOSX Operating System Tests  pts/macosx
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the MacOSX Operating System.
+#### Windows Operating System Tests  pts/windows
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Windows Operating System.
+#### Hurd Operating System Tests  pts/hurd
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Hurd Operating System.
+#### System Subsystem Tests  pts/system
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the System sub-system.
+#### Processor Subsystem Tests  pts/processor
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Processor sub-system.
+#### Graphics Subsystem Tests  pts/graphics
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Graphics sub-system.
+#### Other Subsystem Tests  pts/other
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Other sub-system.
+#### Utility Tests  pts/utility
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Utility software test.
+#### Simulator Tests  pts/simulator
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Simulator software test.
+#### Scientific Tests  pts/scientific
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Scientific software test.
+#### Benchmark Tests  pts/benchmark
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Benchmark software test.
+#### Application Tests  pts/application
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Application software test.
+#### Game Tests  pts/game
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Game software test.
+#### Smp Tests  pts/smp
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing smp.
+#### Cuda Tests  pts/cuda
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing cuda.
+#### Openmp Tests  pts/openmp
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing openmp.
+#### Python Tests  pts/python
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing python.
+#### Go Tests  pts/go
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing go.
+#### Mpi Tests  pts/mpi
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing mpi.
+#### Vdpau Tests  pts/vdpau
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing vdpau.
+#### Video Tests  pts/video
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing video.
+#### Responsiveness Tests  pts/responsiveness
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing responsiveness.
+#### Openmpi Tests  pts/openmpi
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing openmpi.
+#### All SYSTEM Tests  system/all
+This is a collection of all test profiles found within the specified OpenBenchmarking.org repository.
+#### Installed Tests  system/installed
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository that are already installed on the system under test.
+#### Linux Operating System Tests  system/linux
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Linux Operating System.
+#### Solaris Operating System Tests  system/solaris
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Solaris Operating System.
+#### BSD Operating System Tests  system/bsd
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the BSD Operating System.
+#### MacOSX Operating System Tests  system/macosx
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the MacOSX Operating System.
+#### Windows Operating System Tests  system/windows
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Windows Operating System.
+#### Hurd Operating System Tests  system/hurd
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being compatible with the Hurd Operating System.
+#### System Subsystem Tests  system/system
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the System sub-system.
+#### Processor Subsystem Tests  system/processor
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Processor sub-system.
+#### Disk Subsystem Tests  system/disk
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Disk sub-system.
+#### Graphics Subsystem Tests  system/graphics
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Graphics sub-system.
+#### Memory Subsystem Tests  system/memory
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Memory sub-system.
+#### Network Subsystem Tests  system/network
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Network sub-system.
+#### Other Subsystem Tests  system/other
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a test of the Other sub-system.
+#### Utility Tests  system/utility
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Utility software test.
+#### Simulator Tests  system/simulator
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Simulator software test.
+#### Scientific Tests  system/scientific
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Scientific software test.
+#### Benchmark Tests  system/benchmark
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Benchmark software test.
+#### Application Tests  system/application
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Application software test.
+#### Game Tests  system/game
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified as being a Game software test.
+#### Opencl Tests  system/opencl
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing opencl.
+#### Cuda Tests  system/cuda
+This is a collection of test profiles found within the specified OpenBenchmarking.org repository where the test profile is specified via an internal tag as testing cuda.
+
+# Component Testing
+
+## Compiler Testing & Masking
+A majority of the test profiles provided by OpenBenchmarking.org to the Phoronix Test Suite are source-based tests. Relying upon the upstream source-code for each program under test allows for the tests to be easily brought to new platforms and architectures, avoids any out-of-tree / non-default packaging differences by different distributions and operating systems, and to allow the Phoronix Test Suite user to easily test new compilers and/or compiler options. For the source-based tests, the Phoronix Test Suite relies upon a compiler (e.g. GCC, LLVM/Clang, Sun Studio, Open64, et al) being present on the system under test. The Phoronix Test Suite does respect *CC/CXX* environment variables and test profiles are expected to honor *CFLAGS/CXXFLAGS* and other compiler settings.
+As of Phoronix Test Suite 3.8, a compiler masking method is utilized for logging compiler options and other settings with each test profile installation. Prior to installing an open-source test, the Phoronix Test Suite determines the intended compiler to be used based upon the pre-set environment variables or the pre-set compiler(s) within the *PATH* environment variable. The Phoronix Test Suite then masks the compiler to ensure that any options/flags submitted to the compiler are first passed through pts-core so that they can be logged for later use, then is linked to the original, intended compiler. Additionally, other compiler binary names of the same type are blacklisted to prevent their un-intended use (i.e. if a test profile has hard-coded *gcc* in its build script, but *clang* is set as the compiler via *CC* , a sym-link will automatically be created from *gcc* to the masked *clang* for the duration of its test installation).
+The logged compiler data is then used by the Phoronix Test Suite following the test execution process for automated result analysis. If there is a detected change in compiler settings, the differences are automatically reported to the test result graphs. Additionally, key compiler information (e.g. the compiler optimization level and key libraries that were linked at compile-time) is reported as a footnote on result graphs. The Phoronix Test Suite handles all of this in a fully automated manner; test profiles require no changes to take advantage of these compiler-reporting capabilities.
+Separately, the Phoronix Test Suite attempts to automatically log the system compiler's build configuration (i.e. GCC's *gcc -v* "Configured with") output. If the Phoronix Test Suite detects there is a compiler build configuration change between test runs in a result file, it will report each compiler's build configuration to the system information table within the results viewer. Reported to the table is a reduced view of the build configuration options, with less relevant items being stripped away from this view to reduce verbosity. Upon clicking the text, the raw compiler information output can be viewed in full.
+
+## Disk / File-System Testing
+By default tests are installed to *~/.phoronix-test-suite/installed-tests/* . However, the location can be updated from *~/.phoronix-test-suite/user-config.xml* or dynamically via the *PTS_TEST_INSTALL_ROOT_PATH* environment variable.
+When any disk tests are executed, the Phoronix Test Suite will attempt to log the mount options and scheduler of the disk/partition being used for testing. This information is subsequently displayed within the system information table. If the scheduler / mount options are maintained the same throughout all test runs, only a single line is displayed otherwise the options for each test run will be automatically displayed. The file-system in use is always captured and shown in the system information table.
+
+# Phoromatic
+
+## Overview
+The Phoronix Test Suite now features its own built-in Phoromatic server and client implementations. See the *phoromatic.pdf* / *phoromatic.html* documentation for more information on this advanced capability.
+
+# Creating Test Profiles
+
+## Creating A New Phoronix Test Suite / OpenBenchmarking.org Test Profile
+A test profile itself consists of a main XML file containing the test's meta-data ( *test-definition.xml* ), an XML file if needed to specify required files, download URLs and their MD5/SHA256 hashes and file sizes ( *downloads.xml* ), and the *install.sh* script for installing the test and generating the run-script for execution by the Phoronix Test Suite at run-time. There are also other potential files like *pre.sh* , *interim.sh* , and *post.sh* for execution at pre-run, interim-run, and post-run stages by the Phoronix Test Suite. With the script files comprising test profiles, there is also the option of post-fixing them with e.g. *_linux* or *_windows* if wishing to supply different script files based upon the operating system being run during testing, if the setup steps may differ greatly based upon MacOS/Windows/Linux/BSD/Solaris platforms.
+Traditionally the easiest way to learn/understand test profile development has been by looking at looking at a basic test like *c-ray* , *tesseract* or the hundreds of others that are publicly available. Simply run for example *phoronix-test-suite benchmark tesseract* and then look at the test profile's contents either via the default locations in *~/.phoronix-test-suite/test-profiles* or */var/lib/phoronix-test-suite/test-profiles/* if running as root on non-Windows platforms.
+
+### Versioning Mandate
+The Phoronix Test Suite / OpenBenchmarking.org philosophy *mandates that with any change, a new version of the test profile be tagged* . This is done for reproducibility and being able to ensure the exact test profile state when a set of tests are conducted. Test profiles are versioned in a **test-X.Y.Z** format where *X* or *Y* are bumped whenever a change is made to the test profile that makes the results incomparable to a previous version of the tests (e.g. updating against a new upstream code-base, adjusting the parameters to what is benchmarked, etc). The Phoronix Test Suite then knows not to attempt any comparisons with an incompatible version difference or to go and fetch that specific version of the package. The *Z* is bumped when just making non-important changes such as just updating the test's meta-data, changing download URLs, etc. This is easy to enforce with the OpenBenchmarking.org infrastructure rather than having to worry about non-human-friendly Git hashes as test profile versions or having to create Git tags after every commit. The test profile version can optionally be specified when running a test, e.g. *phoronix-test-suite benchmark scimark2-1.2.1* instead of *phoronix-test-suite benchmark scimark2* , which would by default choose the latest available test profile version from OpenBenchmarking.org or the latest version on any local Phoromatic Server. The test profile versions are also always written out as part of the Phoronix Test Suite result XML data.
+
+### Writing A Sample Program
+Writing a test profile for the Phoronix Test Suite is a relatively quick and easy process for anyone familiar with common Linux commands and the basics of XML. To help you understand the design of the Phoronix Test Suite, this guide covers the steps needed to write a testing profile for a very simple application.
+The first step in the profile writing process is to, well, have a piece of software you'd like to use with the Phoronix Test Suite. This software can be closed-source or open-source and be virtually anything as long as it is compatible with an operating system that is supported by the Phoronix Test Suite.
+For this guide, the piece of software being used for demonstration is just a simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz formula. Below is this sample piece of software intended just for demonstration purposes.
+
+> #include &lt;iostream&gt;  
+> #include &lt;math.h&gt;  
+> int main()  
+> {  
+> double pi = 0;  
+>  for(long int i = 1; i &lt;= 87654321; i++)  
+> pi += (double) pow(-1, i + 1) / (2 * i - 1);  
+>  pi *= 4;  
+> std::cout &lt;&lt; &quot;Done Calculating Pi...&quot; &lt;&lt; endl;  
+> return 0;  
+> }
+
+The first step in the actual profile writing process is to name it. If you're  looking to ultimately push this profile to be included in the Phoronix Test Suite, its name must be all lower case and consist of just alpha-numeric characters, but can contain dashes (-). A more advanced test profile capability is operating system prefixes, and if using those there is an underscore separating the prefix from the normal profile name. For this sample profile, we're calling it *sample-program* and the file-name would be *sample-program/test-definition.xml* . Our (very basic) profile is showcased below.
+
+> &lt;PhoronixTestSuite&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestProfile&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Version&gt;1.1.0&lt;/Version&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestType&gt;Processor&lt;/TestType&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;SoftwareType&gt;Utility&lt;/SoftwareType&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;License&gt;FREE&lt;/License&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Status&gt;PRIVATE&lt;/Status&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Maintainer&gt;Phoronix Media&lt;/Maintainer&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/TestProfile&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestInformation&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Title&gt;Sample Pi Program&lt;/Title&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TimesToRun&gt;3&lt;/TimesToRun&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ResultScale&gt;Seconds&lt;/ResultScale&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Proportion&gt;LIB&lt;/Proportion&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Description&gt;A simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz formula. This test can be used for showcasing how to write a basic test profile.&lt;/Description&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ExternalDependencies&gt;build-utilities&lt;/ExternalDependencies&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/TestInformation&gt;  
+> &lt;/PhoronixTestSuite&gt;
+
+This XML profile is what interfaces with the Phoronix Test Suite and provides all the needed information about the test as well as other attributes. For a complete listing of all the supported profile options, look at the specification files in the documentation folder. In the case of *sample-program* , it lets the Phoronix Test Suite know that it's composed of free software, is designed to test the processor, is intended for private use only, and this profile is maintained by Phoronix Media. In addition, it tells the Phoronix Test Suite to execute this program three times and as no result quantifier is set, the average of the three runs will be taken. This profile also tells the Phoronix Test Suite that the generic *build-utilities* package is needed, which will attempt to ensure that default system C/C++ compiler and the standard development utilities/libraries are installed on your Linux distribution. This is needed as the C++ source-code will need to be built from source.
+The next step is to write the *install.sh* file, which once called by the Phoronix Test Suite is intended to install the test locally for benchmarking purposes. The *install.sh* file is technically optional, but is generally used by all tests. Note: The first argument supplied to the install script is the directory that the test needs to be installed to. The *install.sh* file (in our instance) is to be placed inside *test-profiles/sample-program* . Below is the *install.sh* for the *sample-program* .
+
+> #!/bin/sh  
+> tar -xjf sample-pi-program-1.tar.bz2  
+> g++ sample-pi-program.cpp -o sample-pi-program  
+> echo &quot;#!/bin/sh  
+> ./sample-pi-program 2&gt;&amp;1  
+> &quot; &gt; sample-program  
+> chmod +x sample-program
+
+This install file builds the code with GCC, and then creates a small script that is run by the Phoronix Test Suite. Where does the source-code come into play? Well, it needs to be downloaded now from a web server. The Phoronix Test Suite has built-in support for managing downloads from multiple servers in a random over, fall-back support if one mirror is done, and verification of MD5 check-sums. Below is the *downloads.xml* file for *sample-program* that covers all of this.
+
+> &lt;PhoronixTestSuite&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Downloads&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Package&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;URL&gt;http://www.phoronix-test-suite.com/benchmark-files/sample-pi-program.cpp&lt;/URL&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;MD5&gt;e90fb790df8d1544696a1439c9b5bd8d&lt;/MD5&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/Package&gt;  
+> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/Downloads&gt;  
+> &lt;/PhoronixTestSuite&gt;
+
+The final step in the profile writing process is to write a parser to strip all information but the reported result from the standard output or *$LOG_FILE* . In the case of a test profile just measuring how long it takes to run, it is as simple as a *results-definition.xml* looking like:
+
+> &lt;?xml version=&quot;1.0&quot;?&gt;  
+> &lt;PhoronixTestSuite&gt;  
+>   &lt;SystemMonitor&gt;  
+>     &lt;Sensor&gt;sys.time&lt;/Sensor&gt;  
+>   &lt;/SystemMonitor&gt;  
+> &lt;/PhoronixTestSuite&gt;
+
+After that, with all the files in their correct locations, just run: *phoronix-test-suite benchmark sample-program* . The Phoronix Test Suite should now handle the rest by installing the test, running the test, and recording the results (if you so choose). There is no additional work that needs to be done for the results to be recorded in the results viewer or even reporting the results to OpenBenchmarking.org. An up-to-date version of this test profile can be run via *phoronix-test-suite benchmark sample-program* and then by looking at the test profile source via *~/.phoronix-test-suite/test-profiles/pts/sample-program** or within */var/lib/phoronix-test-suite/test-profiles/pts/* if running as root.
+
+# Development Credits
+The Phoronix Test Suite is based upon the extensive testing and internal tools developed by [Phoronix.com](http://www.phoronix.com/) since 2004 along with support from leading tier-one computer hardware and software vendors. The principal architects of the Phoronix Test Suite are [Michael Larabel](http://www.michaellarabel.com/) and Matthew Tippett. The phoronix-test-suite, pts_Graph, Phoromatic, Phodevi, tandem_Xml, and nye_Xml are some of the related open-source projects provided by [Phoronix Media](http://www.phoronix-media.com/) .

documentation/stubs/01_install.html

@@ -5,10 +5,12 @@
 <body>
 <h1>Setup Overview</h1>
 <p>The Phoronix Test Suite supports Linux, Apple Mac OS X, Microsoft Windows, Solaris, Hurd, BSD, and other operating system environments. The only Linux distribution-specific code deals with the external dependencies support feature that are set by individual test profiles. If you are not running one of the supported Linux distributions, Solaris, BSD, or Mac OS X, you may need to install a package manually (as instructed by the Phoronix Test Suite) in order for a test to run. An example of an external dependency would be GCC and the OpenGL Utility Toolkit being needed for test profiles that build an OpenGL benchmark from source-code.</p>
-<p>Among the distributions where the Phoronix Test Suite has been officially tested include Ubuntu, Fedora, Mandriva / Mageia, Gentoo, PCLinuxOS, Arch Linux, Pardus, OpenSuSE, Optware, webOS, Zenwalk, CentOS, Red Hat Enterprise Linux, Oracle Linux, Scientific Linux, Debian, Mint, MEPIS, and Amazon Linux EC2.</p>
+<p>Among the distributions where the Phoronix Test Suite has been officially tested include Ubuntu, Fedora, Mandriva / Mageia, Gentoo, PCLinuxOS, Arch Linux, Pardus, OpenSuSE, Optware, webOS, Zenwalk, CentOS, Red Hat Enterprise Linux, Oracle Linux, Scientific Linux, Debian, Mint, MEPIS, Alpine Linux, Void Linux, Intel Clear Linux, and Amazon Linux EC2.</p>
 <p>Among the tested BSD distributions are FreeBSD, PC-BSD, NetBSD, OpenBSD, Debian GNU/kFreeBSD, and DragonflyBSD. Tested Solaris distributions include OpenSolaris, Solaris Express 11, Oracle Solaris 11, OpenIndiana, Illumos, and Nexenta.</p>
 <h1>Dependencies</h1>
-<p>The only required dependency for the Phoronix Test Suite is PHP 5.2 or newer. On Linux distributions, the needed package is commonly called <em>php5-cli</em> or <em>php-cli</em> or <em>php</em>. It is important to note that only PHP for the command-line is needed and not a web server (Apache) or other packages commonly associated with PHP and its usage by web-sites. The PHP5 version required is PHP 5.2+ and can also be found at <a href="http://www.php.net/">www.php.net</a>.</p>
+<p>The only required dependency for the Phoronix Test Suite is PHP 5.3 or newer. On Linux distributions, the needed package is commonly called <em>php5-cli</em> or <em>php-cli</em> or <em>php7</em> or <em>php</em>. It is important to note that only PHP for the command-line is needed and not a web server (Apache) or other packages commonly associated with PHP and its usage by web-sites. The PHP5 version required is PHP 5.3+ and can also be found at <a href="http://www.php.net/">www.php.net</a>. PHP 7 is fully supported by the Phoronix Test Suite as well as HHVM.</p>
+
+<p>For installing PHP on Windows, the <a href="https://www.microsoft.com/web/platform/phponwindows.aspx">Microsoft Web Platform Installer</a> provides an automated alternative that will install PHP into the directory expected by Phoronix Test Suite. Users opting to install PHP manually must extract the files to <em>C:\Program Files (x86)\PHP\</em> or a subdirectory.</p>
 <p>As part of the PHP requirement, the following PHP extensions are required and/or highly recommended in order to take advantage of the Phoronix Test Suite capabilities:</p>
 <ul>
 <li><strong>PHP DOM</strong> is needed for XML operations and must be installed for the Phoronix Test Suite to function.</li>
@@ -46,14 +48,16 @@
 <h2>OpenSolaris Installation</h2>
 <p>The Phoronix Test Suite core (pts-core) is supported on Solaris and OpenSolaris platforms, but some features may not fully function. Additionally, the PTS External Dependencies feature is supported on OpenSolaris using IPS (the Image Packaging System). However, not all test profiles are supported on Solaris / OpenSolaris, and it may be in violation of your end-user agreement to perform such benchmarks on the Sun Solaris operating system. Using the OpenSolaris Image Packaging System, installing <em>SUNWphp524</em> will provide PHP, PHP GD, and related dependencies. The Phoronix Test Suite has also been tested on Oracle Solaris 11 Express, OpenIndiana, StormOS, and Nexenta Core Platform.</p>
 <h2>BSD Installation</h2>
-<p>The Phoronix Test Suite also supports *BSD operating systems. However, like the Solaris support, not all test profiles are compatible with BSD operating systems, but pts-core has been tested against FreeBSD 7.x/8.x/9.x, NetBSD, PC-BSD 7/8/9, DragonFlyBSD 3.0/3.2/3.4/3.6, and OpenBSD 4.4/4.6. The Phoronix Test Suite will also work with Debian GNU/kFreeBSD.</p>
-<p>For DragonFlyBSD, it's a matter of just using <em>pkg_radd</em> with php5, php5-json, php5-dom, and php5-zip or <em>pkgsrc</em>. Under FreeBSD/PC-BSD, PHP can be obtained from the <em>php5</em> and <em>php5-extensions</em> Ports collection.</p>
+<p>The Phoronix Test Suite also supports *BSD operating systems. However, like the Solaris support, not all test profiles are compatible with BSD operating systems, but pts-core has been tested against FreeBSD 7.x/8.x/9.x, NetBSD, PC-BSD 7/8/9, DragonFlyBSD 3.0/3.2/3.4/3.6/4.2, and OpenBSD 4.4/4.6. The Phoronix Test Suite will also work with Debian GNU/kFreeBSD.</p>
+<p>For DragonFlyBSD, it's a matter of just using <em>pkg_radd</em> with php5, php5-json, php5-dom, and php5-zip or <em>pkgsrc</em>. With DragonFlyBSD using dports, the command is <em>pkg install php5 php5-dom php5-zip php5-json php5-simplexml</em>. Under FreeBSD/PC-BSD, PHP can be obtained from the <em>php5</em> and <em>php5-extensions</em> Ports collection.</p>
 <h2>Mac OS X Installation</h2>
 <p>The Phoronix Test Suite is fully supported on Apple's Mac OS X operating system as of Mac OS X 10.5 with improved support under Mac OS X 10.6. PHP ships with Mac OS X by default so it's simply a matter of downloading the Phoronix Test Suite package, extracting it, and running the executable. For tests that rely upon a compiler, Apple's XCode with GCC and LLVM can be utilized.</p>
 <h2>Windows Installation</h2>
-<p>The Phoronix Test Suite support on Windows is in development and will function just fine, but is not yet feature complete compared to the other operating systems support. At this time the Phoronix Test Suite client is dependent upon PHP being installed to <em>C:\Program Files (x86)\PHP\php</em>. The Phoronix Test Suite also uses <a href="http://www.cpuid.com/cpuz.php">CPU-Z</a> for much of the hardware detection support on Windows and is dependent upon CPU-Z being installed to <em>C:\Program Files\CPUID\CPU-Z\cpuz.exe</em> if you wish to utilize this hardware reporting feature. In terms of the Microsoft Windows support, right now the Phoronix Test Suite is developed and solely tested against Microsoft Windows 7 x64 and Microsoft Windows 8 x64.</p>
+<p>The Phoronix Test Suite support on Microsoft Windows is in development and will function just fine, but is not yet feature complete compared to the other operating systems support. Only a small subset of test profiles, mostly related to graphics, are supported. In terms of the version support, right now the Phoronix Test Suite is developed and tested solely against Microsoft Windows 7 x64 and Microsoft Windows 8 x64.</p>
+<p>At this time the Phoronix Test Suite client is dependent upon the <em>php.exe</em> executable being installed to <em>C:\Program Files (x86)\PHP\ </em> or one of its subdirectories (see PHP installation notes above). Once this requirement is met, simply download the Phoronix Test Suite package, extract it to a directory of your choice, and run <em>phoronix-test-suite.bat</em></p>
+<p>NOTE: because of lack of external dependencies support, it is necessary to manually download and install the files for some test profiles, such as the Unigine graphics benchmarks. See Troubleshooting for more information.</p>
 <h2>Facebook HipHop</h2>
-<p>The Phoronix Test Suite can use Facebook's <a href="https://github.com/facebook/hiphop-php">HopHop HHVM</a> virtual machine as an alternative to the de facto PHP implementation. The Phoronix Test Suite has been tested against HHVM 2.0. If HHVM is present on the system but not PHP, it will automatically be used by the Phoronix Test Suite. Alternatively, the <em>PHP_BIN</em> environment variable can be set to point to an alternative <em>hhvm</em> binary.</p>
+<p>The Phoronix Test Suite can use Facebook's <a href="https://github.com/facebook/hiphop-php">HipHop HHVM</a> virtual machine as an alternative to the de facto PHP implementation. The Phoronix Test Suite has been tested against HHVM 2.0. If HHVM is present on the system but not PHP, it will automatically be used by the Phoronix Test Suite. Alternatively, the <em>PHP_BIN</em> environment variable can be set to point to an alternative <em>hhvm</em> binary.</p>
 <p>The Phoronix Test Suite also supports the older version of Facebook HipHop that serves as a PHP to C++ translator and compiler with GCC. This support though is primitive and not officially supported. To compile the Phoronix Test Suite using HipHop/GCC, run <em>find . -name "*.php" > files.list && hphp --input-list=files.list -k 1 --log=3 --include-path="." --cluster-count=50 -v "AllDynamic=true" -v "AllVolatile=true"</em> from the root <em>phoronix-test-suite/</em> directory. It can then be executed in the form of <em>/tmp/hphp_XXX/program -f pts-core/phoronix-test-suite.php system-info</em>.</p>
 </body>
 </html>

documentation/stubs/02_external_dependencies.html

@@ -1 +1 @@
-<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><title>External Dependencies</title></head><body><p>The Phoronix Test Suite has a feature known as "External Dependencies" where the Phoronix Test Suite can attempt to automatically install some of the test-specific dependencies on supported distributions. If running on a distribution where there is currently no External Dependencies profile, the needed package name(s) are listed for manual installation.</p><p>Below are a list of the operating systems that currently have external dependencies support within the Phoronix Test Suite for the automatic installation of needed test files.<hr><ul></ul><li>Amazon</li><li>Angstrom</li><li>Arch Linux</li><li>CentOS</li><li>ClearOS</li><li>ClearOS Core Server</li><li>Debian</li><li>DragonFlyBSD</li><li>Fedora</li><li>Fluxbuntu</li><li>FreeBSD</li><li>GNU KFreeBSD</li><li>Gentoo</li><li>Goobuntu</li><li>HP</li><li>Joli Cloud</li><li>Linaro</li><li>Linux Mint</li><li>MEPIS</li><li>Mac OS X</li><li>MacPorts</li><li>Mageia</li><li>Mandriva</li><li>MeeGo</li><li>Moblin</li><li>Mythbuntu</li><li>NetBSD</li><li>Nexenta Core</li><li>OLPC</li><li>OpenIndiana</li><li>OpenMandriva</li><li>OpenSolaris</li><li>OpenSuSE</li><li>Optware</li><li>Oracle Server</li><li>PCLinuxOS</li><li>PTS Desktop Live</li><li>PTS Linux Live</li><li>Palm</li><li>Pardus Linux</li><li>Red Hat Enterprise</li><li>Red Hat Enterprise Server</li><li>SUSE</li><li>SUSE Linux</li><li>Scientific</li><li>ScientificSL</li><li>Ubuntu</li><li>Zenwalk</li><li>gNewSense</li></p></body></html>
+<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><title>External Dependencies</title></head><body><p>The Phoronix Test Suite has a feature known as "External Dependencies" where the Phoronix Test Suite can attempt to automatically install some of the test-specific dependencies on supported distributions. If running on a distribution where there is currently no External Dependencies profile, the needed package name(s) are listed for manual installation.</p><p>Below are a list of the operating systems that currently have external dependencies support within the Phoronix Test Suite for the automatic installation of needed test files.<hr><ul></ul><li>Alpine Linux</li><li>Amazon</li><li>Angstrom</li><li>Arch Linux</li><li>CentOS</li><li>ClearOS</li><li>ClearOS Core Server</li><li>Debian</li><li>DragonFlyBSD</li><li>Fedora</li><li>Fluxbuntu</li><li>GNU KFreeBSD</li><li>Gentoo</li><li>Goobuntu</li><li>HP</li><li>Joli Cloud</li><li>Linaro</li><li>Linux Embedded Development Environment</li><li>Linux Mint</li><li>MEPIS</li><li>Mac OS X</li><li>MacPorts</li><li>Mageia</li><li>Mandriva</li><li>MeeGo</li><li>Microsoft Windows</li><li>MidnightBSD</li><li>Moblin</li><li>Mythbuntu</li><li>NetBSD</li><li>Nexenta Core</li><li>OLPC</li><li>OpenIndiana</li><li>OpenMandriva</li><li>OpenSolaris</li><li>OpenSuSE</li><li>Optware</li><li>Oracle Server</li><li>PCLinuxOS</li><li>PTS Desktop Live</li><li>PTS Linux Live</li><li>Palm</li><li>Pardus Linux</li><li>Red Hat Enterprise</li><li>Red Hat Enterprise Server</li><li>SUSE</li><li>SUSE Linux</li><li>Scientific</li><li>ScientificSL</li><li>Solus</li><li>Solus Linux</li><li>Termux</li><li>Ubuntu</li><li>Void Linux</li><li>Zenwalk</li><li>gNewSense</li><li>macOS Brew</li></p></body></html>

documentation/stubs/40_configuration.html

@@ -31,6 +31,8 @@
 <p>For setting any test option(s) from an environment variable rather than being prompted for the options when running a test. Example: <em>PRESET_OPTIONS=&quot;stream.run-type=Add&quot; ./phoronix-test-suite benchmark stream</em>. Multiple options can be passed to this environment variable when delimited by a semicolon.</p>
 <p><strong>SKIP_TESTS</strong></p>
 <p>If there are any test(s) to exempt from the testing process, specify them in this variable. Multiple tests can be waived by delimiting each test identifier by a comma. A test hardware type (i.e. Graphics) can also be supplied for skipping a range of tests.</p>
+<p><strong>SKIP_TESTS_HAVING_ARGS</strong></p>
+<p>If any of the test(s) have an argument matching any strings contained in this environment variable, the test execution will be skipped. Multiple strings can be set when delimiting by a comma.</p>
 <p><strong>RUN_TESTS_IN_RANDOM_ORDER</strong></p>
 <p>Setting this environment variable will cause the tests to be run in a random order.</p>
 <p><strong>SKIP_TESTING_SUBSYSTEMS</strong></p>
@@ -46,7 +48,7 @@
 <p><strong>PTS_DISPLAY_MODE</strong></p>
 <p>If you wish to load a non-default display mode for a single instance, specify the mode in this variable.</p>
 <p><strong>TOTAL_LOOP_TIME</strong></p>
-<p>When running any test(s), if you would like the test(s) to continue running as a loop until a certain time has been reached, this variable can be used. The value should be the number of minutes to run the testing process before the loop is ended.</p>
+<p>When running any test(s), if you would like the test(s) to continue running as a loop until a certain time has been reached, this variable can be used. The value should be the number of minutes to run the testing process before the loop is ended. The testing will finish whenever the currently active test has finished once the time has elapsed. The minimum value allowed is 10 minutes.</p>
 <p><strong>LIMIT_ELAPSED_TEST_TIME</strong></p>
 <p>If you want to ensure that the time for a given Phoronix Test Suite process doesn't elapse past a certain number of minutes, specify the number of minutes for this environment variable. When the amount of time spent testing exceeds that amount, the testing will end prematurely while still saving the tests that were completed in time.</p>
 <p><strong>TOTAL_LOOP_COUNT</strong></p>
@@ -57,10 +59,14 @@
 <p>This is similar to the FORCE_TIMES_TO_RUN option but will only be used if the test profile's run count is less than this defined value.</p>
 <p><strong>NO_FILE_HASH_CHECKS</strong></p>
 <p>To disable MD5/SHA256 check-sums from being checked when downloading test files, set this variable to 1. This variable used to be known as <em>NO_MD5_CHECKS</em>, which is still honored but was changed to <em>NO_FILE_HASH_CHECKS</em> to reflect other kind of file hash sum checks.</p>
+<p><strong>NO_HTTPS</strong></p>
+<p>Set this environment variable to 1 if you don't wish to use HTTPS download links for test profiles (or the system/network lacks HTTPS support). When enabled, HTTPS links will then be done over HTTP.</p>
 <p><strong>PTS_DOWNLOAD_CACHE</strong></p>
 <p>While non-standard Phoronix Test Suite download caches can be specified within the <em>user-config.xml</em> file, an additional directory to look for potential Phoronix Test Suite download files can be specified by this variable.</p>
 <p><strong>GRAPH_HIGHLIGHT</strong></p>
-<p>If this variable is set with a valid test identifer from a result file whether you are using the <em>refresh-graphs</em> command or any other related to the rendering of test results on a bar graph, the specified test identifier's result will be highlighted in a different color than the other rendered test results. Multiple identifiers can be specified when delimited by a comma.</p>
+<p>If this variable is set with a valid test identifer from a result file whether you are using the <em>refresh-graphs</em> command or any other related to the rendering of test results on a bar graph, the specified test identifier's result will be rendered in a different color than the other test results. Multiple identifiers can be specified when delimited by a comma. Additionally, for each key it is possible to provide the actual color value, or an index in the color palette. Example: &quot;will_be_different,group1a=1,group1b=1,blue=#0000ff&quot;</p>
+<p><strong>TEST_EXEC_PREPEND</strong></p>
+<p>Set this variable to any command/environment variable that you may be passed prepended to the test execution string at runtime.</p>
 <p><strong>VIDEO_MEMORY</strong></p>
 <p>If Phodevi fails to detect the system's video memory capacity or is incorrectly detected, the video memory capacity (in MB) can be specified by this variable.</p>
 <p><strong>OVERRIDE_VIDEO_MODES</strong></p>

documentation/stubs/50_general_information.html

@@ -34,6 +34,13 @@
 <p>- If you want the specified test(s) to run in a loop for a set period of time, use the <em>TOTAL_LOOP_TIME</em> environment variable. For instance, running <em>TOTAL_LOOP_TIME=120 phoronix-test-suite benchmark ffmpeg</em> would keep running the ffmpeg test profile for 120 minutes.</p>
 <p>- If you want the specified test(s) to run in a loop for a set number of times, use the <em>TOTAL_LOOP_COUNT</em> environment variable. For instance, running <em>TOTAL_LOOP_COUNT=3 phoronix-test-suite benchmark ffmpeg</em> would keep running the ffmpeg test profile three times.</p>
 <p>- When any tests are being installed and when tests are being run, a lock is created in the system's temporary directory with the name <em>phoronix-test-suite.active</em> (i.e. <em>/tmp/phoronix-test-suite.active</em>) and is removed upon completion. Thus if you have any system scripts that you wish to run when tests are not running or being installed as to not impact the results, one simple way to handle this is by having the script check for the existence of this lock.</p>
+<h2>Troubleshooting</h2>
+In the event that a test profile fails to install or run, the following general troubleshooting steps may be helpful:
+
+<p>- If a test profile fails immediately after starting, check the test profile's directory in <em>~/.phoronix-test-suite/installed-tests/</em> to confirm that the needed files are present. On platforms without External Dependencies support (Windows), it may be necessary to download the files manually and place them in this directory. If this is the case, you will notice that the "Downloading" phase of test installation completes instantly.</p>
+<p>- Inspect the scripts inside the above test profile's directory and confirm that directories or search paths for the test correspond to those on your system</p>
+<p>- Try running the test profile with the <em>debug-benchmark</em> command, or reinstalling with the <em>debug-install</em> command and make note of any unusual output.
+
 <h2>Configuration</h2>
 <p>- The user configuration options for the Phoronix Test Suite are stored in <em>~/.phoronix-test-suite/user-config.xml</em>. The batch mode options are also stored within this file and those can be adjusted by running <em>phoronix-test-suite batch-setup</em>.</p>
 <p>- The colors, size, and other attributes for the graphs found within the Phoronix Test Suite Results Viewer can be modified via the file <em>~/.phoronix-test-suite/graph-config.json</em>.</p>

documentation/stubs/60_phoromatic.html

@@ -4,14 +4,6 @@
 </head>
 <body>
 <h1>Overview</h1>
-<p><em>Phoromatic is a remote management system for the Phoronix Test Suite. Phoromatic allows the automatic (hence the name Phoromatic) scheduling of tests, remote installation of new tests, and the management of multiple test systems all through an intuitive, easy-to-use web interface. Tests can be scheduled to automatically run on a routine basis across multiple test systems. The test results are then available from this central, secure location.</em></p>
-<h1>Basic Instructions</h1>
-<ol>
-<li>Create a free account at <a href="http://www.phoromatic.com/">Phoromatic.com</a>. Phoromatic can also be installed on a LAN / intranet for professional and enterprise support customers of the Phoronix Test Suite.</li>
-<li>On each test system, install the latest version of the Phoronix Test Suite and then run <em>phoronix-test-suite module-setup phoromatic</em>. During this process it will ask the user for an account code and activiation ID that are shown from the <em>Systems</em> page of your Phoromatic account.</li>
-<li>Phoromatic cannot install external dependencies for a test if the Phoronix Test Suite is not running as root, so it would be recommended to now run <em>phoronix-test-suite install-dependencies all</em>. Alternatively, you can also run <em>phoronix-test-suite install all</em> to install all tests at this time, or begin selectively installing the tests you wish to use. Phoromatic can install tests automatically though, it just cannot install external dependencies when the user is not root.</li>
-<li>When you are all set, run <em>phoronix-test-suite phoromatic.start</em>, which begins the process of the Phoronix Test Suite listening to the Phoromatic server waiting for installation and run commands, etc. The installed system software/hardware is also reported all automatically, etc.</li>
-</ol>
-<p>For more information on Phoromatic and a detailed guide, read <a href="http://www.phoronix.com/vr.php?view=14347">Building A Benchmarking Test Farm With Phoromatic</a>.</p>
+<p>The Phoronix Test Suite now features its own built-in Phoromatic server and client implementations. See the <em>phoromatic.pdf</em> / <em>phoromatic.html</em> documentation for more information on this advanced capability.</p>
 </body>
 </html>

documentation/stubs/69_creating_tests.html

@@ -0,0 +1,72 @@
+<html>
+<head>
+<Title>Creating Test Profiles</Title>
+</head>
+<body>
+<h1>Creating A New Phoronix Test Suite / OpenBenchmarking.org Test Profile</h1>
+<p>A test profile itself consists of a main XML file containing the test's meta-data (<em>test-definition.xml</em>), an XML file if needed to specify required files, download URLs and their MD5/SHA256 hashes and file sizes (<em>downloads.xml</em>), and the <em>install.sh</em> script for installing the test and generating the run-script for execution by the Phoronix Test Suite at run-time. There are also other potential files like <em>pre.sh</em>, <em>interim.sh</em>, and <em>post.sh</em> for execution at pre-run, interim-run, and post-run stages by the Phoronix Test Suite. With the script files comprising test profiles, there is also the option of post-fixing them with e.g. <em>_linux</em> or <em>_windows</em> if wishing to supply different script files based upon the operating system being run during testing, if the setup steps may differ greatly based upon MacOS/Windows/Linux/BSD/Solaris platforms.</p>
+<p>Traditionally the easiest way to learn/understand test profile development has been by looking at looking at a basic test like <em>c-ray</em>, <em>tesseract</em> or the hundreds of others that are publicly available. Simply run for example <em>phoronix-test-suite benchmark tesseract</em> and then look at the test profile's contents either via the default locations in <em>~/.phoronix-test-suite/test-profiles</em> or <em>/var/lib/phoronix-test-suite/test-profiles/</em> if running as root on non-Windows platforms.</p>
+<h2>Versioning Mandate</h2>
+<p>The Phoronix Test Suite / OpenBenchmarking.org philosophy <em>mandates that with any change, a new version of the test profile be tagged</em>. This is done for reproducibility and being able to ensure the exact test profile state when a set of tests are conducted. Test profiles are versioned in a <strong>test-X.Y.Z</strong> format where <em>X</em> or <em>Y</em> are bumped whenever a change is made to the test profile that makes the results incomparable to a previous version of the tests (e.g. updating against a new upstream code-base, adjusting the parameters to what is benchmarked, etc). The Phoronix Test Suite then knows not to attempt any comparisons with an incompatible version difference or to go and fetch that specific version of the package. The <em>Z</em> is bumped when just making non-important changes such as just updating the test's meta-data, changing download URLs, etc. This is easy to enforce with the OpenBenchmarking.org infrastructure rather than having to worry about non-human-friendly Git hashes as test profile versions or having to create Git tags after every commit. The test profile version can optionally be specified when running a test, e.g. <em>phoronix-test-suite benchmark scimark2-1.2.1</em> instead of <em>phoronix-test-suite benchmark scimark2</em>, which would by default choose the latest available test profile version from OpenBenchmarking.org or the latest version on any local Phoromatic Server. The test profile versions are also always written out as part of the Phoronix Test Suite result XML data.</p>
+<h2>Writing A Sample Program</h2>
+<p>Writing a test profile for the Phoronix Test Suite is a relatively quick and easy process for anyone familiar with common Linux commands and the basics of XML. To help you understand the design of the Phoronix Test Suite, this guide covers the steps needed to write a testing profile for a very simple application.</p>
+<p>The first step in the profile writing process is to, well, have a piece of software you'd like to use with the Phoronix Test Suite. This software can be closed-source or open-source and be virtually anything as long as it is compatible with an operating system that is supported by the Phoronix Test Suite.</p>
+<p>For this guide, the piece of software being used for demonstration is just a simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz formula. Below is this sample piece of software intended just for demonstration purposes.</p>
+<blockquote>#include &lt;iostream&gt;<br>
+#include &lt;math.h&gt;</p>
+<p>int main()<br>
+{<br>
+double pi = 0;</p>
+<p> for(long int i = 1; i &lt;= 87654321; i++)<br>
+pi += (double) pow(-1, i + 1) / (2 * i - 1);</p>
+<p> pi *= 4;<br>
+std::cout &lt;&lt; &quot;Done Calculating Pi...&quot; &lt;&lt; endl;<br>
+return 0;<br>
+}</blockquote>
+<p>The first step in the actual profile writing process is to name it. If you're  looking to ultimately push this profile to be included in the Phoronix Test Suite, its name must be all lower case and consist of just alpha-numeric characters, but can contain dashes (-). A more advanced test profile capability is operating system prefixes, and if using those there is an underscore separating the prefix from the normal profile name. For this sample profile, we're calling it <em>sample-program</em> and the file-name would be <em>sample-program/test-definition.xml</em>. Our (very basic) profile is showcased below.</p>
+<blockquote>&lt;PhoronixTestSuite&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestProfile&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Version&gt;1.1.0&lt;/Version&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestType&gt;Processor&lt;/TestType&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;SoftwareType&gt;Utility&lt;/SoftwareType&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;License&gt;FREE&lt;/License&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Status&gt;PRIVATE&lt;/Status&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Maintainer&gt;Phoronix Media&lt;/Maintainer&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/TestProfile&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestInformation&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Title&gt;Sample Pi Program&lt;/Title&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TimesToRun&gt;3&lt;/TimesToRun&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ResultScale&gt;Seconds&lt;/ResultScale&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Proportion&gt;LIB&lt;/Proportion&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Description&gt;A simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz formula. This test can be used for showcasing how to write a basic test profile.&lt;/Description&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ExternalDependencies&gt;build-utilities&lt;/ExternalDependencies&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/TestInformation&gt;<br>
+&lt;/PhoronixTestSuite&gt;</blockquote>
+<p>This XML profile is what interfaces with the Phoronix Test Suite and provides all the needed information about the test as well as other attributes. For a complete listing of all the supported profile options, look at the specification files in the documentation folder. In the case of <em>sample-program</em>, it lets the Phoronix Test Suite know that it's composed of free software, is designed to test the processor, is intended for private use only, and this profile is maintained by Phoronix Media. In addition, it tells the Phoronix Test Suite to execute this program three times and as no result quantifier is set, the average of the three runs will be taken. This profile also tells the Phoronix Test Suite that the generic <em>build-utilities</em> package is needed, which will attempt to ensure that default system C/C++ compiler and the standard development utilities/libraries are installed on your Linux distribution. This is needed as the C++ source-code will need to be built from source.</p>
+<p>The next step is to write the <em>install.sh</em> file, which once called by the Phoronix Test Suite is intended to install the test locally for benchmarking purposes. The <em>install.sh</em> file is technically optional, but is generally used by all tests. Note: The first argument supplied to the install script is the directory that the test needs to be installed to. The <em>install.sh</em> file (in our instance) is to be placed inside <em>test-profiles/sample-program</em>. Below is the <em>install.sh</em> for the <em>sample-program</em>.</p>
+<blockquote>#!/bin/sh<br /><br />
+tar -xjf sample-pi-program-1.tar.bz2<br />
+g++ sample-pi-program.cpp -o sample-pi-program<br />
+echo &quot;#!/bin/sh<br />
+./sample-pi-program 2&gt;&amp;1<br />
+&quot; &gt; sample-program<br>
+chmod +x sample-program</blockquote>
+<p>This install file builds the code with GCC, and then creates a small script that is run by the Phoronix Test Suite. Where does the source-code come into play? Well, it needs to be downloaded now from a web server. The Phoronix Test Suite has built-in support for managing downloads from multiple servers in a random over, fall-back support if one mirror is done, and verification of MD5 check-sums. Below is the <em>downloads.xml</em> file for <em>sample-program</em> that covers all of this.</p>
+<blockquote>&lt;PhoronixTestSuite&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Downloads&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Package&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;URL&gt;http://www.phoronix-test-suite.com/benchmark-files/sample-pi-program.cpp&lt;/URL&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;MD5&gt;e90fb790df8d1544696a1439c9b5bd8d&lt;/MD5&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/Package&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/Downloads&gt;<br>
+&lt;/PhoronixTestSuite&gt;</blockquote>
+<p>The final step in the profile writing process is to write a parser to strip all information but the reported result from the standard output or <em>$LOG_FILE</em>. In the case of a test profile just measuring how long it takes to run, it is as simple as a <em>results-definition.xml</em> looking like:</p>
+<blockquote>&lt;?xml version="1.0"?&gt;<br>
+&lt;PhoronixTestSuite&gt;<br>
+  &lt;SystemMonitor&gt;<br>
+    &lt;Sensor&gt;sys.time&lt;/Sensor&gt;<br>
+  &lt;/SystemMonitor&gt;<br>
+&lt;/PhoronixTestSuite&gt;</blockquote>
+<p>After that, with all the files in their correct locations, just run: <em>phoronix-test-suite benchmark sample-program</em>. The Phoronix Test Suite should now handle the rest by installing the test, running the test, and recording the results (if you so choose). There is no additional work that needs to be done for the results to be recorded in the results viewer or even reporting the results to OpenBenchmarking.org. An up-to-date version of this test profile can be run via <em>phoronix-test-suite benchmark sample-program</em> and then by looking at the test profile source via <em>~/.phoronix-test-suite/test-profiles/pts/sample-program*</em> or within <em>/var/lib/phoronix-test-suite/test-profiles/pts/</em> if running as root.</p>
+</body>
+</html>

documentation/stubs/69_pts_os.html

@@ -1,20 +0,0 @@
-<html>
-<head>
-<Title>PTS Desktop Live</Title>
-</head>
-<body>
-<h1>PTS Desktop Live 2010.1 &quot;Anzhofen&quot;</h1>
-<p>PTS Desktop Live and PTS Netbook Live are Linux-based operating systems designed to provide a free software platform for carrying out system tests and hardware benchmarks in an automated, reproducible, and effective manner. The Phoronix Test Suite operating systems are designed around the <a href="http://www.phoronix-test-suite.com/">Phoronix Test Suite</a> with the operating system itself consisting of a stripped down Linux environment, then built upon that is the Phoronix Test Suite with its test profiles and suites. This is not a standard Linux desktop with the normal throng of packages.</p>
-<p>Testing with PTS Desktop Live is intended to be via a Live DVD or Live USB (i.e. flash drive or memory card) environment. PTS Netbook Live is similar to PTS Desktop Live but designed for testing on lightweight x86 systems and netbooks. Both PTS Desktop Live and PTS Netbook Live make it very easy to deploy the Phoronix Test Suite across an array of systems, isolates any software differences when seeking to perform hardware comparisons, and is designed to be very easy to carry out these Linux benchmarks even for a first-time Linux user.</p>
-<p>The inaugural release of this operating system was PTS Desktop Live 2009.3 (codenamed &quot;Gernlinden&quot;) and was based upon the Phoronix Test Suite 2.0 release. The Gernlinden release shipped with more than 50 open-source test profiles and all needed files being located on the disc image. PTS Desktop Live uses the GTK2 GUI dialog for the Phoronix Test Suite by default, but the CLI options are also accessible. PTS Desktop Live also features optimizations to the Linux kernel and related components for use on the latest high-end hardware and other performance related tweaking. PTS Netbook Live features optimizations for the Intel Atom processor.</p>
-<p>The mission of the Phoronix Test Suite and related products will not be achieved until it has made Linux benchmarking incredibly robust and innovative, drop dead easy to perform nearly any kind of benchmark, and when it out-paces other leading operating systems for its benchmarking abilities. These operating systems are being updated quarterly in conjunction with new Phoronix Test Suite releases. The Phoronix Test Suite will continue to be supported under all other Linux distributions, OpenSolaris, Mac OS X, Windows, and *BSD operating systems, but this is just another addition on top of that.</p>
-<p>PTS Desktop Live is designed to run on x86_64 compatible systems with 2GB or more of system memory (ideally 4GB+ if you wish to run many of the tests in a live environment) being highly recommended along with a modern CPU and graphics processor. No proprietary graphics drivers are shipped with PTS Desktop Live at this time, so depending upon the hardware you may not have OpenGL acceleration to run the few graphics-related tests that are included with this release. Only graphics drivers are provided for ATI/AMD, NVIDIA, and Intel hardware. An Internet connection is not required but is needed for communication with OpenBenchmarking.org and for utilizing some of the other Phoronix Test Suite features.</p>
-<p>PTS Netbook Live is designed for netbooks and nettop computers with an Intel Atom processor.</p>
-<h2>Quick Start Guide</h2>
-<p>When the operating system has fully booted, the end-user agreement for the Phoronix Test Suite will be displayed, followed by initializing the main GTK user interface for the Phoronix Test Suite. The Phoronix Test Suite has more than 100 test profiles while there are about 50 test profiles featured within live environment, which are free software, do not pose significant install size problems, and are suitable for benchmarking from a live environment. By default, PTS Desktop Live and PTS Netbook Live will only show the test profiles and suites where all of the needed test files and dependencies ship with this open-source operating system. However, to access any of the test profiles that may require downloading additional files or external dependencies, use the <em>View</em> menu to toggle the different options. The system's hardware and software information can also be viewed in a graphical window from this menu.</p>
-<p>From the <em>Available Tests</em> tab, any test/suite can be selected to view more information. If you wish to install the test, simply click on the install button while waiting shortly for it to install. Afterwards, the test can be run by selecting it again and clicking on the run/benchmark button. If you have a <a href="http://www.openbenchmarking.org/">OpenBenchmarking.org</a> account, you can login from the <em>File</em> menu while Phoronix Test Suite settings are adjustable from the <em>Edit</em> menu.</p>
-<p>When the testing has completed, the graphical user interface will reappear. Via the buttons that appear at the top of the interface, the test results can then be opened within the web browser. Depending upon the test profile, it may also be possible to immediately compare the test result(s) against a set of reference systems. If the completed test(s) do have reference results available, a list of the comparable systems are shown, so that you can click on one or more of these reference systems to see how the given system performs in comparison. Results from OpenBenchmarking.org can also be compared with any of the test results.</p>
-<p>Assuming you did not launch any commands from the terminal to alter the disk(s) or its contents or carry out any other changes; PTS Desktop Live will have not touched your data or the installed operating system(s). If you wish to archive any of your test results, upload the results to OpenBenchmarking.org. Alternatively, when going to the <em>Test Results</em> tab and selecting a test, the <em>File</em> menu presents options for exporting results to text, CSV, and PDF files. The contents of the <em>~/.phoronix-test-suite/test-results/</em> folder can also be backed up for preserving the test results.</p>
-<p>If you wish to use the Phoronix Test Suite from the command-line, open a terminal and run <em>phoronix-test-suite</em>.</p>
-</body>
-</html>

documentation/stubs/99_development_credits.html

@@ -3,55 +3,4 @@
 <Title>Development Credits</Title>
 </head>
 <body>
-<p>The Phoronix Test Suite is based upon the extensive testing and internal tools developed by <a href="http://www.phoronix.com/">Phoronix.com</a> since 2004 along with support from leading tier-one computer hardware and software vendors. The principal architects of the Phoronix Test Suite are <a href="http://www.michaellarabel.com/">Michael Larabel</a> and Matthew Tippett. The phoronix-test-suite, pts_Graph, bilde_renderer, Phodevi, tandem_Xml, and nye_Xml are some of the related open-source projects provided by <a href="http://www.phoronix-media.com/">Phoronix Media</a>.</p>
-<p>Below is a list of individuals and organizations that have contributed upstream patches, test profiles, and/or made other contributions to the Phoronix Test Suite that Phoronix Media would like to thank.</p>
-<ul>
-<li>Achim Gottinger</li>
-<li>Achim Lange</li>
-<li>Andrew Schofield</li>
-<li>Apaige</li>
-<li>Ben Kero</li>
-<li>Bryan Quigley</li>
-<li>Chris Campbell</li>
-<li>Daniel D&iacute;az</li>
-<li>Daniel Stodden / Citrix</li>
-<li>Dean Hilkewich</li>
-<li>ESET s.r.o.</li>
-<li>Fatima Sheremetyeva</li>
-<li>Fujitsu</li>
-<li>Hans Ulrich Niedermann</li>
-<li>Intel Open-Source Technology Center</li>
-<li>Jeroen van Rijn</li>
-<li>Johannes Obermayr</li>
-<li>Juan Aguado</li>
-<li>J&ouml;rg Schirottke</li>
-<li>Kenney Phillis</li>
-<li>Kim Bisgaard</li>
-<li>Laminar Research</li>
-<li>Marco Poletti</li>
-<li>Matthew Tippett</li>
-<li>Michael Larabel</li>
-<li>Michael Opdenacker</li>
-<li>Michael Ploujnikov</li>
-<li>Michal Zatloukal</li>
-<li>Mufasa72</li>
-<li>Nicholas Omann</li>
-<li>NickZ</li>
-<li>Nils Kneuper</li>
-<li>NVIDIA Corporation</li>
-<li>Pekka Panula</li>
-<li>Peter Kraus</li>
-<li>Rainer K&ouml;nig</li>
-<li>SOMA Networks, Inc.</li>
-<li>Stefan Doesinger</li>
-<li>Stepan Hrbek</li>
-<li>St&eacute;phane Teletch&eacute;a</li>
-<li>Sun Microsystems</li>
-<li>Tao Zhang</li>
-<li>Thomas Klausner</li>
-<li>Unigine Corp</li>
-<li>Vadim Peretokin</li>
-<li>Zachary Powers</li>
-</ul>
-</body>
-</html>
+<p>The Phoronix Test Suite is based upon the extensive testing and internal tools developed by <a href="http://www.phoronix.com/">Phoronix.com</a> since 2004 along with support from leading tier-one computer hardware and software vendors. The principal architects of the Phoronix Test Suite are <a href="http://www.michaellarabel.com/">Michael Larabel</a> and Matthew Tippett. The phoronix-test-suite, pts_Graph, Phoromatic, Phodevi, tandem_Xml, and nye_Xml are some of the related open-source projects provided by <a href="http://www.phoronix-media.com/">Phoronix Media</a>.</p>

documentation/stubs/readme-basics.txt

@@ -1,27 +1,25 @@
 The Phoronix Test Suite itself is an open-source framework for conducting automated tests along with reporting of test results, detection of installed system software/hardware, and other features. Modules for the Phoronix Test Suite also allow for integration with git-bisect and other revision control systems for per-commit regression testing, system sensor monitoring, and other extras.
 
-This framework is designed to be an extensible architecture so that new test profiles and suites can be easily added to represent performance benchmarks, unit tests, and other quantitative and qualitative (e.g. image quality comparison) measurements. Available through OpenBenchmarking.org, a collaborative storage platform developed in conjunction with the Phoronix Test Suite, are more than 200 individual test profiles and more than 60 test suites available by default from the Phoronix Test Suite. Independent users are also able to upload their test results, profiles, and suites to OpenBenchmarking.org. A test profile is a single test that can be executed by the Phoronix Test Suite -- with a series of options possible within every test -- and a test suite is a seamless collection of test profiles and/or additional test suites. A test profile consists of a set of bash/shell scripts and XML files while a test suite is a single XML file.
+This framework is designed to be an extensible architecture so that new test profiles and suites can be easily added to represent performance benchmarks, unit tests, and other quantitative and qualitative (e.g. image quality comparison) measurements. Available through OpenBenchmarking.org, a collaborative storage platform developed in conjunction with the Phoronix Test Suite, are more than 200 individual test profiles and more than 60 test suites available by default from the Phoronix Test Suite. Independent users are also able to upload their test results, profiles, and suites to OpenBenchmarking.org. A test profile is a single test that can be executed by the Phoronix Test Suite -- with a series of options possible within every test -- and a test suite is a seamless collection of test profiles and/or additional test suites. A test profile consists of a set of Bash/shell scripts and XML files while a test suite is a single XML file.
 
-OpenBenchmarking.org <http://www.openbenchmarking.org/> also allows for conducting side-by-side result comparisons, a central location for storing and sharing test results, and collaborating over test data. Phoromatic <http://www.phoromatic.com/> is a complementary platform to OpenBenchmarking.org and the Phoronix Test Suite for interfacing with Phoronix Test Suite client(s) to automatically execute test runs on a timed, per-commit, or other trigger-driven basis. Phoromatic is designed for enterprise and allows for the easy management of multiple networked systems running Phoronix Test Suite clients via a single web-based interface.
+[OpenBenchmarking.org](http://www.openbenchmarking.org/) also allows for conducting side-by-side result comparisons, a central location for storing and sharing test results, and collaborating over test data. [Phoromatic](http://www.phoromatic.com/) is a complementary platform to OpenBenchmarking.org and the Phoronix Test Suite for interfacing with Phoronix Test Suite client(s) to automatically execute test runs on a timed, per-commit, or other trigger-driven basis. Phoromatic is designed for enterprise and allows for the easy management of multiple networked systems running Phoronix Test Suite clients via a single web-based interface.
 
 Professional support and custom engineering for the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org is available by contacting <http://www.phoronix-test-suite.com/>.
 
-Full details on the Phoronix Test Suite setup and usage is available from the included HTML/PDF documentation within the phoronix-test-suite package and from the Phoronix Test Suite web-site.
+Full details on the Phoronix Test Suite setup and usage is available from the included HTML/PDF documentation within the phoronix-test-suite package and from the Phoronix Test Suite website.
 
-INSTALLATION & SETUP:
-#####################
+## Installation & Setup
 
 The Phoronix Test Suite is supported on Linux, *BSD, Solaris, Mac OS X, and Windows systems. However, the most full-featured and well supported operating system for conducting the tests is Linux with some non-basic functionality not being available under all platforms. The Phoronix Test Suite software/framework is compatible with all major CPU architectures (e.g. i686, x86_64, ARM, PowerPC), but not all of the test profiles/suites are compatible with all architectures.
 
-The Phoronix Test Suite can be installed for system-wide usage or run locally without installation from the extracted tar.gz/zip package. The only hard dependency on the Phoronix Test Suite is having command-line support for PHP (PHP 5.2+) installed. A complete PHP stack (e.g. with web-server) is NOT needed, but merely the PHP command-line support, which is widely available from operating system package managers under the name php, php5-cli, or php5.
+The Phoronix Test Suite can be installed for system-wide usage or run locally without installation from the extracted tar.gz/zip package. The only hard dependency on the Phoronix Test Suite is having command-line support for PHP (PHP 5.3+) installed. A complete PHP stack (e.g. with web server) is **not** needed, but merely the PHP command-line support, which is widely available from operating system package managers under the name `php`, `php5-cli`, or `php5`.
 
-USAGE:
-######
+## Usage
 
-The process to download, install/setup, execute, and report the results of a benchmark can be as simple as a command such as *phoronix-test-suite benchmark smallpt* to run a simple CPU test profile. If wishing to simply install a test, it's a matter of running *phoronix-test-suite install <test or suite name>* and to run it's *phoronix-test-suite run <test or suite name>*. There's also a batch mode for non-interactive benchmarking by first running *phoronix-test-suite batch-setup* and then using the *batch-run* sub-command rather than *run*.
+The process to download, install/setup, execute, and report the results of a benchmark can be as simple as a command such as `phoronix-test-suite benchmark smallpt` to run a simple CPU test profile. If wishing to simply install a test, it's a matter of running `phoronix-test-suite install <test or suite name>` and to run it's `phoronix-test-suite run <test or suite name>`. There's also a batch mode for non-interactive benchmarking by first running `phoronix-test-suite batch-setup` and then using the `batch-run` sub-command rather than `run`.
 
-Viewing installed system hardware and software is available via *phoronix-test-suite system-info* or *phoronix-test-suite detailed-system-info* for greater verbosity.
+Viewing installed system hardware and software is available via `phoronix-test-suite system-info`.
 
-Facilitating a result comparison from OpenBenchmarking.org can be done by running, for example, *phoronix-test-suite benchmark 1204293-BY-PHORONIX357* if wishing to compare the results of the *http://openbenchmarking.org/result/1204293-BY-PHORONIX357* result file.
+Facilitating a result comparison from OpenBenchmarking.org can be done by running, for example, `phoronix-test-suite benchmark 1204293-BY-PHORONIX357` if wishing to compare the results of the `http://openbenchmarking.org/result/1204293-BY-PHORONIX357` result file.
 
-Additional information is available from the Phoronix Test Suite web-site <http://www.phoronix-test-suite.com/> and the material bundled within the phoronix-test-suite/documentation/ directory. A man page is also bundled with the phoronix-test-suite software.
+Additional information is available from the Phoronix Test Suite website <http://www.phoronix-test-suite.com/> and the material bundled within the `phoronix-test-suite/documentation/` directory or via <http://www.phoronix-test-suite.com/documentation/>. A man page is also bundled with the phoronix-test-suite software.

install-sh

@@ -2,7 +2,7 @@
 
 # Phoronix Test Suite
 # URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-# Copyright (C) 2008 - 2014, Phoronix Media
+# Copyright (C) 2008 - 2018, Phoronix Media
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -42,7 +42,7 @@ fi
 mkdir -p $DESTDIR$INSTALL_PREFIX
 if [ ! -w $DESTDIR$INSTALL_PREFIX ]
 then
-	echo "\nERROR: $DESTDIR$INSTALL_PREFIX is not writable. Run this installer as root or specify a different directory prefix as the first argument sent to this script.\n"
+	echo "ERROR: $DESTDIR$INSTALL_PREFIX is not writable. Run this installer as root or specify a different directory prefix as the first argument sent to this script."
 	exit
 fi
 
@@ -57,8 +57,10 @@ mkdir -p $DESTDIR$INSTALL_PREFIX/share/man/man1/
 mkdir -p $DESTDIR$INSTALL_PREFIX/share/phoronix-test-suite/
 mkdir -p $DESTDIR$INSTALL_PREFIX/share/doc/phoronix-test-suite/
 mkdir -p $DESTDIR$INSTALL_PREFIX/../etc/bash_completion.d/
+#mkdir -p $DESTDIR$INSTALL_PREFIX/../usr/lib/systemd/system/
+#mkdir -p $DESTDIR$INSTALL_PREFIX/../etc/init/
 
-cp CHANGE-LOG $DESTDIR$INSTALL_PREFIX/share/doc/phoronix-test-suite/
+cp ChangeLog $DESTDIR$INSTALL_PREFIX/share/doc/phoronix-test-suite/
 cp COPYING $DESTDIR$INSTALL_PREFIX/share/doc/phoronix-test-suite/
 cp AUTHORS $DESTDIR$INSTALL_PREFIX/share/doc/phoronix-test-suite/
 
@@ -74,8 +76,15 @@ cp pts-core/static/phoronix-test-suite.desktop $DESTDIR$INSTALL_PREFIX/share/app
 cp pts-core/static/phoronix-test-suite-launcher.desktop $DESTDIR$INSTALL_PREFIX/share/applications/
 cp pts-core/static/phoronix-test-suite.appdata.xml $DESTDIR$INSTALL_PREFIX/share/appdata/
 
-rm -f $DESTDIR$INSTALL_PREFIX/share/phoronix-test-suite/pts/etc/scripts/package-build-*
+mkdir -p $DESTDIR$INSTALL_PREFIX/../usr/lib/systemd/system/
+cp deploy/*-systemd/*.service $DESTDIR$INSTALL_PREFIX/../usr/lib/systemd/system/
+
+# mkdir -p $DESTDIR$INSTALL_PREFIX/../etc/init/
+# cp pts-core/static/upstart/*.conf $DESTDIR$INSTALL_PREFIX/../etc/init/
+
+rm -rf $DESTDIR$INSTALL_PREFIX/share/phoronix-test-suite/pts-core
 cp -r pts-core $DESTDIR$INSTALL_PREFIX/share/phoronix-test-suite/
+cp -r deploy $DESTDIR$INSTALL_PREFIX/share/phoronix-test-suite/
 rm -f $DESTDIR$INSTALL_PREFIX/share/phoronix-test-suite/pts-core/static/phoronix-test-suite.desktop
 rm -f $DESTDIR$INSTALL_PREFIX/share/phoronix-test-suite/pts-core/static/phoronix-test-suite-launcher.desktop
 rm -f $DESTDIR$INSTALL_PREFIX/share/phoronix-test-suite/pts-core/openbenchmarking.org/openbenchmarking-mime.xml
@@ -90,7 +99,7 @@ chmod +x $DESTDIR$INSTALL_PREFIX/bin/phoronix-test-suite
 # sed 's:\$url = PTS_PATH . \"documentation\/index.html\";:\$url = \"'"$INSTALL_PREFIX"'\/share\/doc\/packages\/phoronix-test-suite\/index.html\";:g' pts-core/commands/gui_gtk.php > $DESTDIR$INSTALL_PREFIX/share/phoronix-test-suite/pts-core/commands/gui_gtk.php
 
 # XDG MIME OpenBenchmarking support
-if [ "X$DESTDIR" = "X" ]
+if [ "X$DESTDIR" = "X" ] && which xdg-mime >/dev/null && which xdg-icon-resource >/dev/null
 then
 	#No chroot
 	xdg-mime install pts-core/openbenchmarking.org/openbenchmarking-mime.xml

install.bat

@@ -0,0 +1,33 @@
+@echo off
+
+:: Phoronix Test Suite
+:: URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+:: Copyright (C) 2018, Phoronix Media
+::
+:: This program is free software; you can redistribute it and/or modify
+:: it under the terms of the GNU General Public License as published by
+:: the Free Software Foundation; either version 3 of the License, or
+:: (at your option) any later version.
+::
+:: This program is distributed in the hope that it will be useful,
+:: but WITHOUT ANY WARRANTY; without even the implied warranty of
+:: MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+:: GNU General Public License for more details.
+::
+:: You should have received a copy of the GNU General Public License
+:: along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+:: Generic Phoronix Test Suite installer for Windows
+
+:: Ensure the user is in the correct directory
+If Not Exist "pts-core\phoronix-test-suite.php" (
+echo "To install the Phoronix Test Suite you must first change directories to phoronix-test-suite. For support visit: http://www.phoronix-test-suite.com/"
+exit
+)
+set destination="C:\phoronix-test-suite"
+md %destination%
+:: cd /d %destination%
+:: for /F "delims=" %%i in ('dir /b') do (rmdir "%%i" /s/q || del "%%i" /s/q)
+
+xcopy %cd% %destination% /E
+echo Phoronix Test Suite installed to %destination%

phoronix-test-suite

@@ -3,8 +3,8 @@
 #
 #	Phoronix Test Suite
 #	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-#	Copyright (C) 2008 - 2014, Phoronix Media
-#	Copyright (C) 2008 - 2014, Michael Larabel
+#	Copyright (C) 2008 - 2018, Phoronix Media
+#	Copyright (C) 2008 - 2018, Michael Larabel
 #	phoronix-test-suite: The Phoronix Test Suite is an extensible open-source testing / benchmarking platform
 #
 #	This program is free software; you can redistribute it and/or modify
@@ -25,37 +25,85 @@
 # export PTS_DIR=$(readlink -f `dirname $0`)
 export PTS_DIR=`pwd`
 export PTS_MODE="CLIENT"
+if which realpath >/dev/null 2>&1 ;
+then
+	export PTS_LAUNCHER=`realpath $0`
+else
+	export PTS_LAUNCHER="$PTS_DIR/phoronix-test-suite"
+fi
 
 if [ $PTS_DIR != "`pwd`" ]
 then
 	cd $PTS_DIR
 fi
 
+# Run any start-up hooks, such as to pre-seed environment variables or other non-default behaviors / customizations
+if [ -d pts-core/hooks/startup ]
+then
+	for file in pts-core/hooks/startup/*.sh
+	do
+		./$file
+	done
+fi
+
 # Determine PHP binary location
 if [ ! "X$PHP_BIN" = "X" ] && [ -x $PHP_BIN ]
 then
 	export PHP_BIN=$PHP_BIN
-elif [ -x /usr/bin/php5 ] || [ -x /usr/local/bin/php5 ] || [ -x /usr/pkg/bin/php5 ]
+elif [ -x /usr/bin/php7 ] || [ -x /usr/local/bin/php7 ] || [ -x /usr/pkg/bin/php7 ]
 then
-	export PHP_BIN="php5"
-elif [ -x /usr/bin/php ] || [ -x /usr/local/bin/php ] || [ -x /usr/pkg/bin/php ]
+	export PHP_BIN="php7"
+elif [ -x /usr/bin/php ] || [ -x /usr/pkg/bin/php ]
 then
 	export PHP_BIN="php"
-elif [ -x /usr/php5/bin/php ]
+elif [ -x /usr/php7/bin/php ]
 then
-	export PHP_BIN="/usr/php5/bin/php"
+	export PHP_BIN="/usr/php7/bin/php"
 elif [ -x /usr/php/bin/php ]
 then
 	export PHP_BIN="/usr/php/bin/php"
+elif [ -x /usr/local/bin/php ]
+then
+	export PHP_BIN="/usr/local/bin/php"
+elif [ -x /usr/local/bin/php-7.0 ]
+then
+	export PHP_BIN="/usr/local/bin/php-7.0"
+elif [ -x /usr/local/bin/php-7.1 ]
+then
+	export PHP_BIN="/usr/local/bin/php-7.1"
+elif [ -x /usr/local/bin/php-7.2 ]
+then
+	export PHP_BIN="/usr/local/bin/php-7.2"
 elif [ -x /opt/bin/php ]
 then
 	export PHP_BIN="/opt/bin/php"
 elif [ -x /usr/pkg/libexec/cgi-bin/php ]
 then
 	export PHP_BIN="/usr/pkg/libexec/cgi-bin/php"
+elif which php >/dev/null 2>&1 ;
+then
+	export PHP_BIN="php"
+elif which php7 >/dev/null 2>&1 ;
+then
+	export PHP_BIN="php7"
+elif [ -x /usr/php5/bin/php ]
+then
+	export PHP_BIN="/usr/php5/bin/php"
+elif [ -x /usr/bin/php5 ] || [ -x /usr/local/bin/php5 ] || [ -x /usr/pkg/bin/php5 ]
+then
+	export PHP_BIN="php5"
+elif which php5 >/dev/null 2>&1 ;
+then
+	export PHP_BIN="php5"
+elif which php-cli >/dev/null 2>&1 ;
+then
+	export PHP_BIN="php-cli"
 elif [ -x /usr/bin/hhvm ]
 then
 	export PHP_BIN="/usr/bin/hhvm"
+elif which hhvm >/dev/null 2>&1 ;
+then
+	export PHP_BIN="hhvm"
 else
 	export PHP_BIN=""
 fi
@@ -65,42 +113,136 @@ if [ "X$PHP_BIN" = "X" ]
 then
 	cat <<'EOT'
 
-PHP 5.2 or newer must be installed for the Phoronix Test Suite
+PHP 5.3 or newer or PHP 7 must be installed for the Phoronix Test Suite
 The PHP command-line package is commonly called php-cli, php5-cli, or php.
 For more information visit: http://www.phoronix-test-suite.com/
 
 EOT
 
-if [ -x /usr/bin/apt-get ]
+if [ -x /usr/bin/zypper ]
 then
+	# SUSE / OpenSUSE
 	echo "The command to likely run for your operating system is: "
-	echo "# apt-get install php5-cli php5-json php5-gd"
+	echo "# zypper install php7 php7-zip"
+	echo " "
+elif [ -x /usr/bin/apt-get ]
+then
+	# Debian / Ubuntu Based Systems
+	echo "The command to likely run for your operating system is: "
+	echo "# apt-get install php-cli php-xml"
+	echo " "
+elif [ -x /usr/bin/dnf ]
+then
+	# Modern Fedora
+	echo "The command to likely run for your operating system is: "
+	echo "# dnf install php-cli php-xml"
 	echo " "
 elif [ -x /usr/bin/yum ]
 then
+	# Red Hat / Fedora / etc
 	echo "The command to likely run for your operating system is: "
-	echo "# yum install php-cli php-xml"
-	echo " "
-elif [ -x /usr/bin/zypper ]
-then
-	echo "The command to likely run for your operating system is: "
-	echo "# zypper install php5 php5-zip"
+	echo "# yum install php-cli php-xml php-json"
 	echo " "
 elif [ -x /usr/bin/pkg_radd ]
 then
+	# BSD
 	echo "The command to likely run for your operating system is: "
-	echo "# pkg_radd php5 php5-json php5-zip php5-dom"
+	echo "# pkg_radd php7 php7-json php7-zip php7-dom"
 	echo " "
 elif [ -x /usr/bin/pacman ]
 then
+	# Arch Linux
 	echo "The command to likely run for your operating system is: "
 	echo "# pacman -S php"
 	echo " "
+elif [ -x /usr/local/sbin/pkg ] || [ -x /usr/sbin/pkg ]
+then
+	# DragonFlyBSD and others
+	echo "The command to likely run for your operating system is: "
+	echo "# pkg install php72 php72-dom php72-zip php72-json php72-simplexml php72-openssl"
+	echo " "
+elif [ -x /usr/local/swupd ]
+then
+	# Clear Linux
+	echo "The command to likely run for your operating system is: "
+	echo "# swupd bundle-add os-testsuite-phoronix"
+	echo " "
+elif [ -x /usr/sbin/pkg_add ]
+then
+	# OpenBSD
+	echo "The command to likely run for your operating system is: "
+	echo "# pkg_add php php-zip"
+	echo " "
+elif [ -x /usr/sbin/equo ]
+then
+	# Sabayon
+	echo "The command to likely run for your operating system is: "
+	echo "# equo install dev-lang/php"
+	echo " "
+elif [ -x /usr/bin/xbps-install ] || [ -x /usr/sbin/xbps-install ]
+then
+	# Void Linux
+	echo "The command to likely run for your operating system is: "
+	echo "# xbps-install -Sy php unzip"
+	echo " "
+elif [ -x /usr/sbin/netpkg ]
+then
+	# Zenwalk / Slackware
+	echo "The command to likely run for your operating system is: "
+	echo "# netpkg php"
+	echo " "
+elif [ -x /sbin/apk ]
+then
+	# Alpine Linux
+	echo "The command to likely run for your operating system is: "
+	echo "# apk add php7 php7-dom php7-zip php7-json php7-simplexml"
+	echo " "
+elif [ -x /usr/bin/urpmi ]
+then
+	# OpenMandriva Linux
+	echo "The command to likely run for your operating system is: "
+	echo "# urpmi php-cli"
+	echo " "
+elif [ -x /usr/bin/eopkg ]
+then
+	# Solus Linux
+	echo "The command to likely run for your operating system is: "
+	echo "# eopkg install php"
+	echo " "
+elif [ -x /usr/sbin/mport ]
+then
+	# MidnightBSD
+	echo "The command to likely run for your operating system is: "
+	echo "# mport install php70 php70-dom php70-zip php70-json php70-simplexml"
+	echo " "
+elif [ -x /bin/opkg ]
+then
+	# Linux Embedded Development Environment
+	echo "The command to likely run for your operating system is: "
+	echo "# opkg install php7-cli php7-mod-dom php7-mod-zip php7-mod-json php7-mod-simplexml"
+	echo " "
+elif [ -x /usr/bin/equo ]
+then
+	# Sabayon
+	echo "The command to likely run for your operating system is: "
+	echo "# equo install php"
+	echo " "
+elif [ -x /usr/bin/emerge ]
+then
+	# Gentoo
+	echo "The command to likely run for your operating system is: "
+	echo "# emerge dev-lang/php"
+	echo " "
 fi
 
 exit
 fi
 
+if [ ! -f pts-core/phoronix-test-suite.php ] && [ -f `dirname $0`/pts-core/phoronix-test-suite.php ]
+then
+	cd `dirname $0`
+fi
+
 # Ensure the user is in the correct directory
 if [ ! -f pts-core/phoronix-test-suite.php ]
 then
@@ -118,15 +260,16 @@ case "$1" in
 "test-module" | "debug-module" | "webui")
 	export PTS_IGNORE_MODULES=1
 	;;
+"debug-self-test")
+	export PTS_IGNORE_MODULES=1
+	export PTS_SILENT_MODE=1
+	;;
+"enterprise-setup")
+	export PTS_SILENT_MODE=1
+	;;
 esac
 
-if [ -f $HOME/.phoronix-test-suite/web-server-launcher ]
-then
-	rm -f $HOME/.phoronix-test-suite/web-server-launcher
-elif [ -f $HOME/.phoronix-test-suite/phoromatic-server-launcher ]
-then
-	rm -f $HOME/.phoronix-test-suite/phoromatic-server-launcher
-fi
+export PTS_EXT_LAUNCH_SCRIPT_DIR=`mktemp -d 2>/dev/null || mktemp -d -t ptstmp`
 
 # Run The Phoronix Test Suite
 PTS_EXIT_STATUS=8
@@ -135,14 +278,15 @@ while [ $PTS_EXIT_STATUS -eq 8 ]; do
 	PTS_EXIT_STATUS=$?
 done
 
-if [ -f $HOME/.phoronix-test-suite/web-server-launcher ]
+if [ -f $PTS_EXT_LAUNCH_SCRIPT_DIR/web-server-launcher ]
 then
-	chmod +x $HOME/.phoronix-test-suite/web-server-launcher
-	exec $HOME/.phoronix-test-suite/web-server-launcher
-	rm -f $HOME/.phoronix-test-suite/web-server-launcher
-elif [ -f $HOME/.phoronix-test-suite/phoromatic-server-launcher ]
+	chmod +x $PTS_EXT_LAUNCH_SCRIPT_DIR/web-server-launcher
+	exec $PTS_EXT_LAUNCH_SCRIPT_DIR/web-server-launcher
+	rm -f $PTS_EXT_LAUNCH_SCRIPT_DIR/web-server-launcher
+elif [ -f $PTS_EXT_LAUNCH_SCRIPT_DIR/phoromatic-server-launcher ]
 then
-	chmod +x $HOME/.phoronix-test-suite/phoromatic-server-launcher
-	exec $HOME/.phoronix-test-suite/phoromatic-server-launcher
-	rm -f $HOME/.phoronix-test-suite/phoromatic-server-launcher
+	chmod +x $PTS_EXT_LAUNCH_SCRIPT_DIR/phoromatic-server-launcher
+	exec $PTS_EXT_LAUNCH_SCRIPT_DIR/phoromatic-server-launcher
+	rm -f $PTS_EXT_LAUNCH_SCRIPT_DIR/phoromatic-server-launcher
 fi
+# rm -rf $PTS_EXT_LAUNCH_SCRIPT_DIR

phoronix-test-suite.bat

@@ -1,7 +1,7 @@
 ::	Phoronix Test Suite
 ::	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-::	Copyright (C) 2008 - 2014, Phoronix Media
-::	Copyright (C) 2008 - 2014, Michael Larabel
+::	Copyright (C) 2008 - 2018, Phoronix Media
+::	Copyright (C) 2008 - 2018, Michael Larabel
 ::	phoronix-test-suite: The Phoronix Test Suite is an extensible open-source testing / benchmarking platform
 ::
 ::	This program is free software; you can redistribute it and/or modify
@@ -20,16 +20,34 @@
 
 :: Full path to root directory of the actual Phoronix Test Suite code
 @echo off
-set PTS_DIR=%cd%
+set PTS_DIR=%~dp0
 set PTS_MODE=CLIENT
+set PTS_LAUNCHER=%0
+
+:: See if php was setup via Cygwin64, the benefit there is allowing access to PCNTL extensions, etc
+:: If exist C:\cygwin64\bin\php.exe (
+:: set PHP_BIN=C:\cygwin64\bin\php.exe
+:: )
 
 :: TODO: Other work to bring this up to sync with the *NIX phoronix-test-suite launcher
 If defined PHP_BIN goto SkipBinSearch
-echo "No PHP_BIN defined checking for usual locations."
-If exist "C:\Program Files (x86)\PHP"\php set PHP_BIN="C:\Program Files (x86)\PHP"\php
-If exist C:\php-gtk2\php set PHP_BIN=C:\php-gtk2\php
-set PHP_BIN="C:\Program Files (x86)\PHP"\php
+  
+:: Download PHP for Windows and then extract it
+If not exist C:\PHP\php.exe (
+echo Attempting to download and setup Windows PHP release.
+If not exist php.zip (
+powershell -command "& { iwr http://phoronix-test-suite.com/benchmark-files/php-7.2.3-Win32-VC15-x64.zip -OutFile php.zip }"
+)
+powershell -command "& { Expand-Archive php.zip -DestinationPath C:\PHP }"
+If not exist VC_redist.x64.exe (
+echo Attempting to download and run Visual C++ Redistributable for Visual Studio 2017 support.
+powershell -command "& { iwr https://go.microsoft.com/fwlink/?LinkId=746572 -OutFile VC_redist.x64.exe }"
+VC_redist.x64.exe
+)
+  )
+:: Use the newly downloaded PHP location
+set PHP_BIN=C:\PHP\php.exe
 
 :SkipBinSearch
-If not defined PHP_BIN echo "\nPHP could not be found.\n" else cls
-%PHP_BIN% pts-core/phoronix-test-suite.php %*
+
+%PHP_BIN% "%PTS_DIR%\pts-core\phoronix-test-suite.php" %*

pts-core/commands/analyze_all_runs.php

@@ -1,30 +1,24 @@
 <?php
-
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
 	Copyright (C) 2008 - 2011, Phoronix Media
 	Copyright (C) 2008 - 2011, Michael Larabel
-
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
 	the Free Software Foundation; either version 3 of the License, or
 	(at your option) any later version.
-
 	This program is distributed in the hope that it will be useful,
 	but WITHOUT ANY WARRANTY; without even the implied warranty of
 	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
 	GNU General Public License for more details.
-
 	You should have received a copy of the GNU General Public License
 	along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
-
 class analyze_all_runs implements pts_option_interface
 {
-	const doc_section = 'Result Analytics';
+	const doc_section = 'Result Management';
 	const doc_description = 'This option will generate a candlestick graph showing the distribution of results from all trial runs. The candlestick graph is similar to the Japanese candlestick charts used by the financial industry, except instead of representing stock data it is numerical result data from all trial runs.\n\nThe tip of the upper-wick represents the highest value of the test runs with the tip of the lower-wick representing the lowest value of all test runs. The upper-edge of the candle body represents the first or last run value and the lower-edge represents the first or last run value. Lastly, if the last run value is less than the first run value, the candle body is the same color as the graph background, otherwise the last run value is greater.';
-
 	public static function argument_checks()
 	{
 		return array(
@@ -33,8 +27,7 @@ class analyze_all_runs implements pts_option_interface
 	}
 	public static function run($args)
 	{
-		pts_client::regenerate_graphs($args[0], 'The ' . $args[0] . ' result file graphs have been re-rendered to show all test runs.', array('graph_render_type' => 'CANDLESTICK'));
+		pts_client::regenerate_graphs($args[0], 'The ' . $args[0] . ' result file graphs have been re-rendered to show all test runs.', array('graph_render_type' => 'HORIZONTAL_BOX_PLOT'));
 	}
 }
-
 ?>

pts-core/commands/analyze_batch.php

@@ -1,54 +0,0 @@
-<?php
-
-/*
-	Phoronix Test Suite
-	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2011, Phoronix Media
-	Copyright (C) 2008 - 2011, Michael Larabel
-
-	This program is free software; you can redistribute it and/or modify
-	it under the terms of the GNU General Public License as published by
-	the Free Software Foundation; either version 3 of the License, or
-	(at your option) any later version.
-
-	This program is distributed in the hope that it will be useful,
-	but WITHOUT ANY WARRANTY; without even the implied warranty of
-	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-	GNU General Public License for more details.
-
-	You should have received a copy of the GNU General Public License
-	along with this program. If not, see <http://www.gnu.org/licenses/>.
-*/
-
-class analyze_batch implements pts_option_interface
-{
-	const doc_section = 'Result Analytics';
-	const doc_description = 'This option will analyze a batch results file and plot out the performance impact from the different options onto a line graph (i.e. to see the impact that changing the video resolution has on the system\'s performance).';
-
-	public static function argument_checks()
-	{
-		return array(
-		new pts_argument_check(0, array('pts_types', 'is_result_file'), null)
-		);
-	}
-	public static function run($args)
-	{
-		$base_file = $args[0];
-
-		do
-		{
-			$rand_file = rand(1000, 9999);
-			$save_to = 'analyze-' . $rand_file . '/';
-		}
-		while(is_dir(PTS_SAVE_RESULTS_PATH . $save_to));
-		$save_to .= 'composite.xml';
-
-		// Analyze Results
-		$SAVED_RESULTS = pts_merge::generate_analytical_batch_xml($base_file);
-		pts_client::save_test_result($save_to, $SAVED_RESULTS);
-		echo 'Results Saved To: ' . PTS_SAVE_RESULTS_PATH . $save_to . PHP_EOL . PHP_EOL;
-		pts_client::display_web_page(PTS_SAVE_RESULTS_PATH . dirname($save_to) . '/index.html');
-	}
-}
-
-?>

pts-core/commands/analyze_image_delta.php

@@ -1,138 +0,0 @@
-<?php
-
-/*
-	Phoronix Test Suite
-	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2011, Phoronix Media
-	Copyright (C) 2009 - 2011, Michael Larabel
-
-	This program is free software; you can redistribute it and/or modify
-	it under the terms of the GNU General Public License as published by
-	the Free Software Foundation; either version 3 of the License, or
-	(at your option) any later version.
-
-	This program is distributed in the hope that it will be useful,
-	but WITHOUT ANY WARRANTY; without even the implied warranty of
-	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-	GNU General Public License for more details.
-
-	You should have received a copy of the GNU General Public License
-	along with this program. If not, see <http://www.gnu.org/licenses/>.
-*/
-
-class analyze_image_delta implements pts_option_interface
-{
-	const doc_section = 'Result Analytics';
-	const doc_description = 'This option will analyze a test result file if it contains any test results that produced an image quality comparison (IQC) and will render image deltas illustrating the difference between images from two test results.';
-
-	public static function argument_checks()
-	{
-		return array(
-		new pts_argument_check(0, array('pts_types', 'is_result_file'), null)
-		);
-	}
-	public static function run($args)
-	{
-		$result = $args[0];
-
-		$result_file = new pts_result_file($result);
-		$result_file_identifiers = $result_file->get_system_identifiers();
-
-		if(count($result_file_identifiers) < 2)
-		{
-			echo PHP_EOL . 'There are not multiple test runs in this result file.' . PHP_EOL;
-			return false;
-		}
-
-		$base_identifier = pts_user_io::prompt_text_menu('Select the base test run', $result_file_identifiers);
-		$base_select = new pts_result_merge_select($result, $base_identifier);
-
-		$compare_identifier = pts_user_io::prompt_text_menu('Select the test run to compare', $result_file_identifiers);
-		$compare_select = new pts_result_merge_select($result, $compare_identifier);
-
-		do
-		{
-			$extract_to = 'iqc-analyze-' . rand(100, 999);
-		}
-		while(is_dir(PTS_SAVE_RESULTS_PATH . $extract_to));
-
-		$extract_result = pts_merge::merge_test_results($base_select, $compare_select);
-		pts_client::save_test_result($extract_to . '/composite.xml', $extract_result);
-
-		$compare_file = new pts_result_file($extract_to);
-		$result_file_writer = new pts_result_file_writer('Image Delta');
-
-		foreach($compare_file->get_result_objects() as $result_object)
-		{
-			if($result_object->test_profile->get_display_format() != 'IMAGE_COMPARISON')
-			{
-				continue;
-			}
-
-			$base_result = null;
-			$compare_result = null;
-
-			foreach($result_object->test_result_buffer->get_buffer_items() as $buffer_item)
-			{
-				if($buffer_item->get_result_identifier() == $base_identifier && $base_result == null)
-				{
-					$base_result = $buffer_item->get_result_value();
-				}
-				else if($buffer_item->get_result_identifier() == $compare_identifier && $compare_result == null)
-				{
-					$compare_result = $buffer_item->get_result_value();
-				}
-
-				if($compare_result != null && $base_result != null)
-				{
-					break;
-				}
-			}
-
-			if($compare_result == null || $base_result == null)
-			{
-				continue;
-			}
-
-			$base_img = imagecreatefromstring(base64_decode($base_result));
-			$compare_img = imagecreatefromstring(base64_decode($compare_result));
-			$delta_img = imagecreatefromstring(base64_decode($compare_result));
-			$img_width = imagesx($base_img);
-			$img_height = imagesy($base_img);
-			$img_changed = false;
-
-			for($x = 0; $x < $img_width; $x++)
-			{
-				for($y = 0; $y < $img_height; $y++)
-				{
-					$base_image_color = pts_image::rgb_gd_color_at($base_img, $x, $y);
-					$compare_image_color = pts_image::rgb_gd_color_at($compare_img, $x, $y);
-
-					if($base_image_color == $compare_image_color || pts_image::rgb_int_diff($base_image_color, $compare_image_color) < 9)
-					{
-						if(($cords = pts_image::color_pixel_delta($base_img, $compare_img, $x, $y)))
-						{
-							$pixel_rgb = pts_image::rgb_gd_color_at($delta_img, $cords[0], $cords[1]);
-							$color_invert = imagecolorresolve($delta_img, 255 - $pixel_rgb[0], 255 - $pixel_rgb[1], 255 - $pixel_rgb[2]);
-							imagesetpixel($delta_img, $x, $y, $color_invert);
-							$img_changed = true;
-						}
-					}
-				}
-			}
-
-			if($img_changed)
-			{
-				imagepng($delta_img, PTS_SAVE_RESULTS_PATH . $extract_to . '/scratch.png');
-				$result_value = base64_encode(file_get_contents(PTS_SAVE_RESULTS_PATH . $extract_to . '/scratch.png', FILE_BINARY));
-				pts_file_io::unlink(PTS_SAVE_RESULTS_PATH . $extract_to . '/scratch.png');
-				$result_file_writer->add_result_from_result_object_with_value_string($result_object, $result_value);
-			}
-		}
-
-		pts_client::save_result_file($result_file_writer, $extract_to);
-		pts_client::display_web_page(PTS_SAVE_RESULTS_PATH . $extract_to . '/composite.xml');
-	}
-}
-
-?>

pts-core/commands/auto_compare.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2012, Phoronix Media
-	Copyright (C) 2012, Michael Larabel
+	Copyright (C) 2012 - 2018, Phoronix Media
+	Copyright (C) 2012 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -25,10 +25,6 @@ class auto_compare implements pts_option_interface
 	const doc_section = 'Testing';
 	const doc_description = 'This option will autonomously determine the most relevant test(s) to run for any selected sub-system(s). The tests to run are determined via OpenBenchmarking.org integration with the global results pool. Related test results from OpenBenchmarking.org are also merged to provide a straight-forward and effective means of carrying out a system comparison. If wishing to find comparable results for any particular test profile(s), simply pass the test profile names as additional arguments to this command.';
 
-	public static function invalid_command($passed_args = null)
-	{
-		pts_tests::invalid_command_helper($passed_args);
-	}
 	public static function run($r)
 	{
 		$compare_tests = array();
@@ -39,7 +35,7 @@ class auto_compare implements pts_option_interface
 
 			if($test_object instanceof pts_test_profile)
 			{
-				array_push($compare_tests, $test_object->get_identifier(false));
+				$compare_tests[] = $test_object->get_identifier(false);
 
 				if(!isset($compare_subsystems[$test_object->get_test_hardware_type()]))
 				{
@@ -113,11 +109,10 @@ class auto_compare implements pts_option_interface
 
 			foreach($json['auto_compare']['public_ids'] as $public_id)
 			{
-				$result_xml = pts_openbenchmarking::clone_openbenchmarking_result($public_id, true);
-
-				if($result_xml)
+				$ret = pts_openbenchmarking::clone_openbenchmarking_result($public_id);
+				if($ret)
 				{
-					$result_file = new pts_result_file($result_xml);
+					$result_file = new pts_result_file($public_id);
 					$result_objects = $result_file->get_result_objects();
 
 					foreach($result_objects as $i => &$result_object)
@@ -140,54 +135,49 @@ class auto_compare implements pts_option_interface
 						continue;
 					}
 					$result_file->override_result_objects($result_objects);
-
-					array_push($compare_results, $result_file);
+					pts_client::save_test_result($result_file->get_file_location(), $result_file->get_xml());
+					$compare_results[] = $public_id;
 				}
 			}
 
 			if(count($compare_results) > 0)
 			{
-				$result_xml = pts_merge::merge_test_results_array($compare_results);
+				$result_file = new pts_result_file(null, true);
+				$result_file->merge($compare_results);
+				$result_objects = $result_file->get_result_objects();
+				$system_count = $result_file->get_system_count();
+				$result_count = count($result_objects);
+				$result_match_count = array();
 
-				if(count($compare_results) > 2)
+				if($result_count > 3)
 				{
-					$result_file = new pts_result_file($result_xml);
-					$result_objects = $result_file->get_result_objects();
-					$system_count = $result_file->get_system_count();
-					$result_count = count($result_objects);
-					$result_match_count = array();
-
-					if($result_count > 3)
+					foreach($result_objects as $i => &$result_object)
 					{
-						foreach($result_objects as $i => &$result_object)
-						{
-							$result_match_count[$i] = $result_object->test_result_buffer->get_count();
-						}
+						$result_match_count[$i] = $result_object->test_result_buffer->get_count();
+					}
 
-						arsort($result_match_count);
-						$biggest_size = pts_arrays::first_element($result_match_count);
-						if($biggest_size == $system_count || $biggest_size > 3)
+					arsort($result_match_count);
+					$biggest_size = pts_arrays::first_element($result_match_count);
+					if($biggest_size == $system_count || $biggest_size > 3)
+					{
+						foreach($result_match_count as $key => $value)
 						{
-							foreach($result_match_count as $key => $value)
+							if($value < 2)
 							{
-								if($value < 2)
-								{
-									unset($result_objects[$key]);
-								}
+								unset($result_objects[$key]);
 							}
 						}
-
-						$result_file->override_result_objects($result_objects);
-						$result_xml = pts_merge::merge_test_results_array(array($result_file));
 					}
+					$result_file->override_result_objects($result_objects);
 				}
 
-				pts_client::save_test_result('auto-comparison/composite.xml', $result_xml);
+				pts_client::save_test_result('auto-comparison/composite.xml', $result_file->get_xml());
 			}
 		}
 
 		pts_test_installer::standard_install(array('auto-comparison'));
-		pts_test_run_manager::standard_run(array('auto-comparison'));
+		$test_run_manager = new pts_test_run_manager();
+		$test_run_manager->standard_run(array('auto-comparison'));
 	}
 	protected static function system_component_to_format(&$system_info, &$to_array, $component_types, $allow_trim_extra = false)
 	{
@@ -203,10 +193,10 @@ class auto_compare implements pts_option_interface
 					{
 						$value_r = explode(' ', str_replace('-', ' ', $value));
 						array_pop($value_r);
-						array_push($to_array, $component_type . ':' . implode(' ', $value_r));
+						$to_array[] = $component_type . ':' . implode(' ', $value_r);
 					}
 
-					array_push($to_array, $component_type . ':' . $value);
+					$to_array[] = $component_type . ':' . $value;
 				}
 			}
 		}

pts-core/commands/auto_load_module.php

@@ -0,0 +1,68 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2017, Phoronix Media
+	Copyright (C) 2017, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class auto_load_module implements pts_option_interface
+{
+	const doc_section = 'Modules';
+	const doc_description = 'This option can be used for easily adding a module to the LoadModules list in the Phoronix Test Suite user configuration file. That list controls what PTS modules are automatically loaded on start-up of the Phoronix Test Suite.';
+
+	public static function run($r)
+	{
+		$loaded_modules = pts_strings::comma_explode(pts_config::read_user_config('PhoronixTestSuite/Options/Modules/LoadModules', null));
+		$available_modules = pts_module_manager::list_available_modules();
+		echo PHP_EOL . 'Currently Loaded Modules: ' . PHP_EOL;
+		echo pts_user_io::display_text_list($loaded_modules);
+		echo PHP_EOL . 'Available Modules: ' . PHP_EOL;
+		echo pts_user_io::display_text_list($available_modules);
+
+
+		if(count($r) == 0)
+		{
+			echo PHP_EOL . 'You must specify a valid module from the list to load.' . PHP_EOL;
+			echo 'Example: phoronix-test-suite auto-load-module update_checker' . PHP_EOL;
+			return false;
+		}
+
+		foreach($r as $module_to_load)
+		{
+			if(!in_array($module_to_load, $available_modules))
+			{
+				echo PHP_EOL . 'Module Not Available: ' . $module_to_load . PHP_EOL;
+			}
+			else if(in_array($module_to_load, $loaded_modules))
+			{
+				echo PHP_EOL . 'Module Already Loaded: ' . $module_to_load . PHP_EOL;
+			}
+			else
+			{
+				echo PHP_EOL . 'Module To Load: ' . $module_to_load . PHP_EOL;
+				array_push($loaded_modules, $module_to_load);
+			}
+		}
+
+		$new_options = array('PhoronixTestSuite/Options/Modules/LoadModules' => implode(', ', $loaded_modules));
+		pts_config::user_config_generate($new_options);
+		echo PHP_EOL . 'New user configuration file written.' . PHP_EOL . PHP_EOL;
+	}
+}
+
+?>

pts-core/commands/auto_sort_result_file.php

@@ -0,0 +1,55 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2014 - 2018, Phoronix Media
+	Copyright (C) 2014 - 2018, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class auto_sort_result_file implements pts_option_interface
+{
+	const doc_section = 'Result Management';
+	const doc_description = 'This option is used if you wish to automatically attempt to sort the results by their result identifier string.';
+
+	public static function argument_checks()
+	{
+		return array(
+		new pts_argument_check(0, array('pts_types', 'is_result_file'), null)
+		);
+	}
+	public static function run($args)
+	{
+		$result_file = new pts_result_file($args[0]);
+		$result_file_identifiers = $result_file->get_system_identifiers();
+
+		if(count($result_file_identifiers) < 2)
+		{
+			echo PHP_EOL . 'There are not multiple test runs in this result file.' . PHP_EOL;
+			return false;
+		}
+
+		$extract_selects = array();
+		echo PHP_EOL . 'Automatically sorting the results...' . PHP_EOL;
+
+		sort($result_file_identifiers);
+		$result_file->reorder_runs($result_file_identifiers);
+		pts_client::save_test_result($result_file->get_file_location(), $result_file->get_xml());
+		pts_client::display_web_page(PTS_SAVE_RESULTS_PATH . $args[0] . '/index.html');
+	}
+}
+
+?>

pts-core/commands/batch_benchmark.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009, Phoronix Media
-	Copyright (C) 2009, Michael Larabel
+	Copyright (C) 2009 - 2015, Phoronix Media
+	Copyright (C) 2009 - 2015, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -33,8 +33,9 @@ class batch_benchmark implements pts_option_interface
 	}
 	public static function run($r)
 	{
-		pts_test_installer::standard_install($r, pts_c::batch_mode);
-		pts_test_run_manager::standard_run($r, pts_c::batch_mode);
+		pts_test_installer::standard_install($r, false, true);
+		$test_run_manager = new pts_test_run_manager(true);
+		$test_run_manager->standard_run($r);
 	}
 }
 

pts-core/commands/batch_install.php

@@ -33,7 +33,7 @@ class batch_install implements pts_option_interface
 	}
 	public static function run($r)
 	{
-		pts_test_installer::standard_install($r, pts_c::batch_mode);
+		pts_test_installer::standard_install($r, false, true);
 	}
 }
 

pts-core/commands/batch_run.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2012, Phoronix Media
-	Copyright (C) 2009 - 2012, Michael Larabel
+	Copyright (C) 2009 - 2018, Phoronix Media
+	Copyright (C) 2009 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -33,11 +33,8 @@ class batch_run implements pts_option_interface
 	}
 	public static function run($r)
 	{
-		pts_test_run_manager::standard_run($r, pts_c::batch_mode);
-	}
-	public static function invalid_command($passed_args = null)
-	{
-		pts_tests::invalid_command_helper($passed_args);
+		$test_run_manager = new pts_test_run_manager(true);
+		$test_run_manager->standard_run($r);
 	}
 }
 

pts-core/commands/benchmark.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2011, Phoronix Media
-	Copyright (C) 2008 - 2011, Michael Larabel
+	Copyright (C) 2008 - 2015, Phoronix Media
+	Copyright (C) 2008 - 2015, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -29,10 +29,6 @@ class benchmark implements pts_option_interface
 	{
 		return array('test');
 	}
-	public static function invalid_command($passed_args = null)
-	{
-		pts_tests::invalid_command_helper($passed_args);
-	}
 	public static function argument_checks()
 	{
 		return array(
@@ -42,7 +38,8 @@ class benchmark implements pts_option_interface
 	public static function run($r)
 	{
 		pts_test_installer::standard_install($r);
-		pts_test_run_manager::standard_run($r);
+		$run_manager = new pts_test_run_manager();
+		$run_manager->standard_run($r);
 	}
 }
 

pts-core/commands/build_suite.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2013, Phoronix Media
-	Copyright (C) 2008 - 2013, Michael Larabel
+	Copyright (C) 2008 - 2017, Phoronix Media
+	Copyright (C) 2008 - 2017, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -22,7 +22,7 @@
 
 class build_suite implements pts_option_interface
 {
-	const doc_section = 'Other';
+	const doc_section = 'Asset Creation';
 	const doc_description = 'This option will guide the user through the process of generating their own test suite, which they can then run. Optionally, passed as arguments can be the test(s) or suite(s) to add to the suite to be created, instead of being prompted through the process.';
 
 	public static function run($r)
@@ -34,11 +34,15 @@ class build_suite implements pts_option_interface
 		$suite_maintainer = pts_user_io::prompt_user_input('Enter suite maintainer name');
 		$suite_description = pts_user_io::prompt_user_input('Enter suite description');
 
-		$possible_suites = pts_openbenchmarking::available_suites();
-		$possible_tests = pts_openbenchmarking::available_tests();
+		$possible_suites = pts_openbenchmarking::available_suites(false);
+		$possible_tests = pts_openbenchmarking::available_tests(false);
 
-		$suite_writer = new pts_test_suite_writer();
-		$suite_writer->add_suite_information($suite_name, '1.0.0', $suite_maintainer, $suite_test_type, $suite_description);
+		$new_suite = new pts_test_suite();
+		$new_suite->set_title($suite_name);
+		$new_suite->set_version('1.0.0');
+		$new_suite->set_maintainer($suite_maintainer);
+		$new_suite->set_suite_type($suite_test_type);
+		$new_suite->set_description($suite_description);
 
 		foreach($r as $test_object)
 		{
@@ -51,12 +55,12 @@ class build_suite implements pts_option_interface
 				for($i = 0; $i < count($args); $i++)
 				{
 					// Not binding the test profile version to this suite, otherwise change false to true
-					$suite_writer->add_to_suite($test_object->get_identifier(false), $args[$i], $description[$i]);
+					$new_suite->add_to_suite($test_object, $args[$i], $description[$i]);
 				}
 			}
 			else if($test_object instanceof pts_test_suite)
 			{
-				$suite_writer->add_to_suite($test_object->get_identifier(), null, null);
+				$new_suite->add_suite_tests_to_suite($test_object);
 			}
 		}
 
@@ -74,12 +78,13 @@ class build_suite implements pts_option_interface
 
 					for($i = 0; $i < count($args); $i++)
 					{
-						$suite_writer->add_to_suite($test_to_add, $args[$i], $description[$i]);
+						$new_suite->add_to_suite($test_profile, $args[$i], $description[$i]);
 					}
 					break;
 				case 'Add Sub-Suite':
 					$suite_to_add = pts_user_io::prompt_text_menu('Enter test suite', $possible_suites);
-					$suite_writer->add_to_suite($suite_to_add, null, null);
+					$test_suite = new pts_test_suite($suite_to_add);
+					$new_suite->add_suite_tests_to_suite($test_suite);
 					break;
 			}
 			echo PHP_EOL . 'Available Options:' . PHP_EOL;
@@ -87,13 +92,9 @@ class build_suite implements pts_option_interface
 		}
 		while($input_option != 'Save & Exit');
 
-		$suite_identifier = pts_test_run_manager::clean_save_name($suite_name);
-		$save_to = PTS_TEST_SUITE_PATH . 'local/' . $suite_identifier . '/suite-definition.xml';
-		mkdir(dirname($save_to));
-
-		if($suite_writer->save_xml($save_to) != false)
+		if($new_suite->save_xml($suite_name) != false)
 		{
-			echo PHP_EOL . PHP_EOL . 'Saved To: ' . $save_to . PHP_EOL . 'To run this suite, type: phoronix-test-suite benchmark ' . $suite_identifier . PHP_EOL . PHP_EOL;
+			echo PHP_EOL . PHP_EOL . 'Saved -- to run this suite, type: phoronix-test-suite benchmark ' . $new_suite->get_identifier() . PHP_EOL . PHP_EOL;
 		}
 	}
 }

pts-core/commands/clone_openbenchmarking_result.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2010, Phoronix Media
-	Copyright (C) 2008 - 2010, Michael Larabel
+	Copyright (C) 2008 - 2015, Phoronix Media
+	Copyright (C) 2008 - 2015, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -24,7 +24,7 @@ class clone_openbenchmarking_result implements pts_option_interface
 {
 	const doc_section = 'OpenBenchmarking.org';
 	const doc_use_alias = 'clone-result';
-	const doc_description = 'This option will download a local copy of a file that was saved to OpenBenchmarking.org, as long as a valid public ID is supplied. More than one ID can be specified and the results will then be merged.';
+	const doc_description = 'This option will download a local copy of a file that was saved to OpenBenchmarking.org, as long as a valid public ID is supplied.';
 
 	public static function command_aliases()
 	{
@@ -42,14 +42,13 @@ class clone_openbenchmarking_result implements pts_option_interface
 		foreach($args as $id)
 		{
 			$xml = pts_openbenchmarking::clone_openbenchmarking_result($id, true);
-			array_push($result_files, new pts_result_file($xml));
+			if($xml)
+			{
+				$result_file = new pts_result_file($xml);
+				pts_client::save_test_result($id . '/composite.xml', $result_file->get_xml(), true);
+				echo PHP_EOL . 'Result Saved To: ' . PTS_SAVE_RESULTS_PATH . $id . '/composite.xml' . PHP_EOL;
+			}
 		}
-
-		$writer = new pts_result_file_writer(null);
-		pts_merge::merge_test_results_process($writer, $result_files);
-		pts_client::save_test_result($args[0] . '/composite.xml', $writer->get_xml(), true);
-
-		echo PHP_EOL . 'Result Saved To: ' . PTS_SAVE_RESULTS_PATH . $args[0] . '/composite.xml' . PHP_EOL;
 	}
 }
 

pts-core/commands/commands.php

@@ -3,9 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2010, Phoronix Media
-	Copyright (C) 2008 - 2010, Michael Larabel
-	bilde_png_renderer: The PNG rendering implementation for bilde_renderer.
+	Copyright (C) 2018, Phoronix Media
+	Copyright (C) 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -21,18 +20,28 @@
 	along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 
-class bilde_png_renderer extends bilde_gd_renderer
+class commands implements pts_option_interface
 {
-	public $renderer = "PNG";
+	const doc_section = 'Other';
+	const doc_description = 'This option will display a short list of possible Phoronix Test Suite commands.';
 
-	public function render_image($output_file = null, $quality = 100)
+	public static function run($r)
 	{
-		$quality = floor(9 - (($quality / 100) * 9)); // calculate compression level
-		if(defined("BILDE_IMAGE_INTERLACING"))
+		$options = pts_documentation::client_commands_array();
+		$commands = array();
+		foreach($options as $section => &$contents)
 		{
-			imageinterlace($this->image, BILDE_IMAGE_INTERLACING);
+			if(empty($contents))
+			{
+				continue;
+			}
+			foreach($contents as &$option)
+			{
+				$commands[] = $option[0];
+			}
 		}
-		return imagepng($this->image, $output_file, $quality);
+		echo pts_user_io::display_packed_list($commands);
+		echo PHP_EOL;
 	}
 }
 

pts-core/commands/copy_run_in_result_file.php

@@ -1,64 +0,0 @@
-<?php
-
-/*
-	Phoronix Test Suite
-	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2010 - 2013, Phoronix Media
-	Copyright (C) 2010 - 2013, Michael Larabel
-
-	This program is free software; you can redistribute it and/or modify
-	it under the terms of the GNU General Public License as published by
-	the Free Software Foundation; either version 3 of the License, or
-	(at your option) any later version.
-
-	This program is distributed in the hope that it will be useful,
-	but WITHOUT ANY WARRANTY; without even the implied warranty of
-	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-	GNU General Public License for more details.
-
-	You should have received a copy of the GNU General Public License
-	along with this program. If not, see <http://www.gnu.org/licenses/>.
-*/
-
-class copy_run_in_result_file implements pts_option_interface
-{
-	const doc_section = 'Result Management';
-	const doc_description = 'This option is used if you wish to change an existing test run within a saved results file.';
-
-	public static function argument_checks()
-	{
-		return array(
-		new pts_argument_check(0, array('pts_types', 'is_result_file'), null)
-		);
-	}
-	public static function run($r)
-	{
-		$result = $r[0];
-
-		$result_file = new pts_result_file($result);
-		$result_file_identifiers = $result_file->get_system_identifiers();
-
-		$copy_identifier = pts_user_io::prompt_text_menu('Select the test run to copy', $result_file_identifiers);
-		$copy_identifier_new = pts_user_io::prompt_user_input('Enter the new identifier of the copied run');
-		$merge_selects = array();
-
-		foreach($result_file_identifiers as $identifier)
-		{
-			$this_merge_select = new pts_result_merge_select($result, $identifier);
-			array_push($merge_selects, $this_merge_select);
-
-			if($identifier == $copy_identifier)
-			{
-				$this_merge_select = new pts_result_merge_select($result, $identifier);
-				$this_merge_select->rename_identifier($copy_identifier_new);
-				array_push($merge_selects, $this_merge_select);
-			}
-		}
-
-		$extract_result = pts_merge::merge_test_results_array($merge_selects);
-		pts_client::save_test_result($r[0] . '/composite.xml', $extract_result);
-		pts_client::display_web_page(PTS_SAVE_RESULTS_PATH . $r[0] . '/index.html');
-	}
-}
-
-?>

pts-core/commands/create_test_profile.php

@@ -0,0 +1,91 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2018, Phoronix Media
+	Copyright (C) 2018, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class create_test_profile implements pts_option_interface
+{
+	const doc_section = 'Asset Creation';
+	const doc_description = 'This option can be used for creating a Phoronix Test Suite test profile by answering questions about the test for constructing the test profile XML meta-data and handling other boiler-plate basics for getting started in developing new tests.';
+
+	public static function run($r)
+	{
+		echo 'The create-test-profile helper will attempt to walk you through creating the basics of creating a new test profile for execution by the Phoronix Test Suite. Follow the prompts to begin filling out the XML meta-data that defines a test profile.' . PHP_EOL;
+
+		do
+		{
+			$is_valid = true;
+			$input = pts_user_io::prompt_user_input('Enter an identifier/name for the test profile', false, false);
+			$input = pts_validation::string_to_sanitized_test_profile_base($input);
+
+			if(pts_test_profile::is_test_profile($input))
+			{
+				$is_valid = false;
+				echo 'There is already a ' . $input . ' test profile.' . PHP_EOL;
+			}
+			else if(pts_test_suite::is_suite($input))
+			{
+				$is_valid = false;
+				echo 'There is already a test suite named ' . $input . '.' . PHP_EOL;
+			}
+			else if(pts_result_file::is_test_result_file($input))
+			{
+				$is_valid = false;
+				echo 'There is already a result file named ' . $input . '.' . PHP_EOL;
+			}
+		}
+		while(!$is_valid);
+
+		$tp_identifier = 'local/'. $input;
+		$tp_path = PTS_TEST_PROFILE_PATH . $tp_identifier;
+		echo 'Creating test profile: ' . $tp_identifier . ' @ ' . $tp_path . PHP_EOL;
+		pts_file_io::mkdir($tp_path);
+
+		$types = pts_validation::process_xsd_types();
+
+		pts_client::$display->generic_heading('test-definition.xml Creation');
+		$writer = new nye_XmlWriter();
+		pts_validation::xsd_to_cli_creator(pts_openbenchmarking::openbenchmarking_standards_path() . 'schemas/test-profile.xsd', $writer, $types);
+		$writer->saveXMLFile($tp_path . '/test-definition.xml');
+		echo 'Generated: ' . $tp_path . '/test-definition.xml' . PHP_EOL;
+
+		pts_client::$display->generic_heading('downloads.xml Creation');
+		$writer = new nye_XmlWriter();
+		do
+		{
+			pts_validation::xsd_to_cli_creator(pts_openbenchmarking::openbenchmarking_standards_path() . 'schemas/test-profile-downloads.xsd', $writer, $types);
+		}
+		while(pts_user_io::prompt_bool_input('Add another file/download?', -1));
+		$writer->saveXMLFile($tp_path . '/downloads.xml');
+		echo 'Generated: ' . $tp_path . '/downloads.xml' . PHP_EOL;
+
+		/*
+		pts_client::$display->generic_heading('results-definition.xml Creation');
+		$writer = new nye_XmlWriter();
+		pts_validation::xsd_to_cli_creator(pts_openbenchmarking::openbenchmarking_standards_path() . 'schemas/results-parser.xsd', $writer, $types);
+		$writer->saveXMLFile($tp_path . '/results-definition.xml');
+		echo 'Generated: ' . $tp_path . '/results-definition.xml' . PHP_EOL;
+		*/
+
+		pts_validation::generate_test_profile_file_templates($tp_identifier, $tp_path);
+	}
+}
+
+?>

pts-core/commands/debug_benchmark.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2010, Phoronix Media
-	Copyright (C) 2009 - 2010, Michael Larabel
+	Copyright (C) 2009 - 2015, Phoronix Media
+	Copyright (C) 2009 - 2015, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -20,7 +20,7 @@
 	along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 
-class debug_run implements pts_option_interface
+class debug_benchmark implements pts_option_interface
 {
 	const doc_section = 'Asset Creation';
 	const doc_description = 'This option is intended for use by test profile writers and is identical to the <em>run</em> option but will yield more information during the run process that can be used to debug issues with a test profile or to verify the test profile is functioning correctly.';
@@ -31,9 +31,20 @@ class debug_run implements pts_option_interface
 		new pts_argument_check('VARIABLE_LENGTH', array('pts_types', 'identifier_to_object'), null)
 		);
 	}
+	public static function command_aliases()
+	{
+		return array('debug_run');
+	}
 	public static function run($r)
 	{
-		pts_test_run_manager::standard_run($r, pts_c::debug_mode);
+		// Make sure you're debugging the latest test script...
+		pts_test_installer::standard_install($r);
+		// For debugging, usually running just once is sufficient, unless FORCE_TIMES_TO_RUN is preset
+		pts_client::pts_set_environment_variable('FORCE_TIMES_TO_RUN', 1);
+		// Run the test(s) in debug mode
+		pts_client::set_debug_mode(true);
+		$test_run_manager = new pts_test_run_manager();
+		$test_run_manager->standard_run($r);
 	}
 }
 

pts-core/commands/debug_dependency_handler.php

@@ -0,0 +1,45 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2015, Phoronix Media
+	Copyright (C) 2015, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class debug_dependency_handler implements pts_option_interface
+{
+	const doc_section = 'Other';
+	const doc_description = 'This option is used for testing the distribution-specific dependency handler for external dependencies.';
+
+	public static function run($r)
+	{
+		$exdep_parser = new pts_exdep_generic_parser();
+		foreach($exdep_parser->get_available_packages() as $pkg)
+		{
+			$pkg_data = $exdep_parser->get_package_data($pkg);
+			$files = explode(' ', str_replace(array(' OR ', ', '), ' ', $pkg_data['file_check']));
+			foreach($files as $file)
+			{
+				echo (is_array($file) ? implode(' ', $file) : $file) . ': ';
+				$deps = pts_external_dependencies::packages_that_provide($file);
+				echo (is_array($deps) ? implode(' ', $deps) : null) . PHP_EOL;
+			}
+		}
+	}
+}
+
+?>

pts-core/commands/debug_install.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2010, Phoronix Media
-	Copyright (C) 2009 - 2010, Michael Larabel
+	Copyright (C) 2009 - 2015, Phoronix Media
+	Copyright (C) 2009 - 2015, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -33,7 +33,8 @@ class debug_install implements pts_option_interface
 	}
 	public static function run($r)
 	{
-		pts_test_installer::standard_install($r, pts_c::debug_mode | pts_c::force_install);
+		pts_client::set_debug_mode(true);
+		pts_test_installer::standard_install($r, true);
 	}
 }
 

pts-core/commands/debug_render_test.php

@@ -0,0 +1,125 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2015 - 2017, Phoronix Media
+	Copyright (C) 2015 - 2017, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class debug_render_test implements pts_option_interface
+{
+	const doc_section = 'Other';
+	const doc_description = 'This option is used during the development of the Phoronix Test Suite software for testing of the result and graph rendering code-paths This option will download a large number of reference test results from LinuxBenchmarking.com.';
+
+	public static function run($r)
+	{
+		$render_dir = pts_client::temporary_directory() . '/pts-render-test-20171112/';
+		if(!is_file($render_dir . 'mega-render-test-20171112.tar.xz'))
+		{
+			pts_file_io::mkdir($render_dir);
+			pts_network::download_file('http://linuxbenchmarking.com/misc/mega-render-test-20171112.tar.xz', $render_dir . 'mega-render-test-20171112.tar.xz');
+		}
+		pts_compression::archive_extract($render_dir . 'mega-render-test-20171112.tar.xz');
+
+		if(!defined('PATH_TO_EXPORTED_PHOROMATIC_DATA'))
+		{
+			define('PATH_TO_EXPORTED_PHOROMATIC_DATA', $render_dir . 'mega-render-test-20171112/');
+		}
+
+		error_reporting(E_ALL);
+		ini_set('memory_limit','2048M');
+
+		$export_index_json = file_get_contents(PATH_TO_EXPORTED_PHOROMATIC_DATA . 'export-index.json');
+		$export_index_json = json_decode($export_index_json, true);
+		$dump_size = 0;
+		$start = microtime(true);
+		foreach(array_keys($export_index_json['phoromatic']) as $REQUESTED)
+		{
+			$this_render_test = time();
+			$tracker = &$export_index_json['phoromatic'][$REQUESTED];
+			$triggers = $tracker['triggers'];
+			echo PHP_EOL . 'STARTING RENDER TEST ON: ' . $REQUESTED . ' (' . count($triggers) . ' Triggers)' . PHP_EOL;
+			$length = count($tracker['triggers']);
+			$result_files = array();
+
+			foreach($triggers as $trigger)
+			{
+				$results_for_trigger = glob(PATH_TO_EXPORTED_PHOROMATIC_DATA . '/' . $REQUESTED . '/' . $trigger . '/*/composite.xml');
+				echo '.';
+				if($results_for_trigger == false)
+					continue;
+
+				foreach($results_for_trigger as $composite_xml)
+				{
+					// Add to result file
+					$system_name = basename(dirname($composite_xml)) . ': ' . $trigger;
+					$result_files[] = new pts_result_merge_select($composite_xml, null, $system_name);
+				}
+			}
+			echo 'STARTING MERGE; ';
+			$result_file = new pts_result_file(null, true);
+			$result_file->merge($result_files);
+			echo 'MAKING NEW RESULT FILE; ';
+			$extra_attributes = array('reverse_result_buffer' => true, 'force_simple_keys' => true, 'force_line_graph_compact' => true, 'force_tracking_line_graph' => true);
+			//$extra_attributes['normalize_result_buffer'] = true;
+
+			$intent = null;
+			//$table = new pts_ResultFileTable($result_file, $intent);
+			//echo '<p style="text-align: center; overflow: auto;" class="result_object">' . pts_render::render_graph_inline_embed($table, $result_file, $extra_attributes) . '</p>';
+			echo 'STARTING RESULT LOOP; ';
+			$html_dump = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html>
+<head>
+<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+</head>
+<body>';
+			foreach($result_file->get_result_objects((isset($_POST['show_only_changed_results']) ? 'ONLY_CHANGED_RESULTS' : -1)) as $i => $result_object)
+			{
+				echo $result_object->test_profile->get_title() . ' ';
+				$html_dump .= '<h3>' . $result_object->get_arguments_description() . '</h3>';
+				$html_dump .= pts_render::render_graph_inline_embed($result_object, $result_file, $extra_attributes);
+				unset($result_object);
+			}
+
+			$table = new pts_ResultFileSystemsTable($result_file);
+			$html_dump .= pts_render::render_graph_inline_embed($table, $result_file, $extra_attributes);
+			echo PHP_EOL . PHP_EOL . 'RENDER TEST ON: ' . $REQUESTED . ' TOOK ' . (time() - $this_render_test) . PHP_EOL;
+			$dump_size += strlen($html_dump);
+			file_put_contents(PATH_TO_EXPORTED_PHOROMATIC_DATA . $REQUESTED . '.html', $html_dump . '</body></html>');
+		}
+
+		if(getenv('DEBUG_RENDER_TEST_LOCAL_TOO') != false)
+		{
+			echo PHP_EOL . 'LOCAL RENDER TEST TIME' . PHP_EOL;
+			$extra_graph_attributes = null;
+			foreach(pts_client::saved_test_results() as $saved_result)
+			{
+				$save_to_dir = pts_client::setup_test_result_directory($saved_result);
+				$generated_graphs = pts_client::generate_result_file_graphs($saved_result, $save_to_dir, $extra_graph_attributes);
+				echo $saved_result . ': ' . count($generated_graphs) . PHP_EOL;
+			}
+		}
+
+		echo PHP_EOL . 'RENDER TEST TOOK: ' . (time() - $start) . PHP_EOL . PHP_EOL;
+		echo PHP_EOL . 'PEAK MEMORY USAGE: ' . round(memory_get_peak_usage(true) / 1048576, 3) . ' MB';
+		echo PHP_EOL . 'PEAK MEMORY USAGE (emalloc): ' . round(memory_get_peak_usage() / 1048576, 3) . ' MB';
+		echo PHP_EOL . 'TOTAL FILE SIZE: ' . ceil($dump_size / 1000) . ' KB';
+		echo PHP_EOL;
+	}
+}
+
+?>

pts-core/commands/debug_result_parser.php

@@ -0,0 +1,48 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2017, Phoronix Media
+	Copyright (C) 2017, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class debug_result_parser implements pts_option_interface
+{
+	const doc_section = 'Asset Creation';
+	const doc_description = 'This option is intended for use by test profile writers and is used for debugging a result parser. No test execution is done, but there must already be PTS-generated .log files present within the test\'s installation directory.';
+
+	public static function argument_checks()
+	{
+		return array(
+		new pts_argument_check('VARIABLE_LENGTH', array('pts_types', 'identifier_to_object'), null)
+		);
+	}
+	public static function run($r)
+	{
+		// Make sure you're debugging the latest test script...
+		//pts_test_installer::standard_install($r);
+		// For debugging, usually running just once is sufficient, unless FORCE_TIMES_TO_RUN is preset
+		pts_client::pts_set_environment_variable('FORCE_TIMES_TO_RUN', 1);
+		// Run the test(s) in debug mode
+		pts_client::set_debug_mode(true);
+		$test_run_manager = new pts_test_run_manager();
+		$test_run_manager->DEBUG_no_test_execution_just_result_parse = true;
+		$test_run_manager->standard_run($r);
+	}
+}
+
+?>

pts-core/commands/debug_self_test.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2013, Phoronix Media
-	Copyright (C) 2013, Michael Larabel
+	Copyright (C) 2013 - 2018, Phoronix Media
+	Copyright (C) 2013 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -27,16 +27,19 @@ class debug_self_test implements pts_option_interface
 
 	public static function run($r)
 	{
+		define('PHOROMATIC_PROCESS', true);
 		$commands = array(
-			'detailed_system_info' => null,
+			'system_info' => null,
 			'list_available_tests' => null,
 			'list_available_suites' => null,
-			'info' => array('xonotic'),
-			'clone_openbenchmarking_result' => array('1107247-LI-MESACOMMI48'),
-		//	'refresh_graphs' => array('1107247-LI-MESACOMMI48'),
+			'info' => array('pts/all'),
+			'clone_openbenchmarking_result' => array('1107247-LI-MESACOMMI48', '1509040-HA-GCCINTELS17', '1508201-HA-GTX95073337', '1508233-HA-INTELSKYL16'),
+			'refresh_graphs' => array('1107247-LI-MESACOMMI48', '1711094-AL-ZOTACGEFO61', '1711073-AL-GTX770TIL45', '1710268-AL-CPUTESTS119'),
 			'result_file_to_text' => array('1107247-LI-MESACOMMI48'),
+			'merge_results' => array('1107247-LI-MESACOMMI48', '1509040-HA-GCCINTELS17', '1508201-HA-GTX95073337', '1508233-HA-INTELSKYL16', '1711094-AL-ZOTACGEFO61', '1711073-AL-GTX770TIL45', '1710268-AL-CPUTESTS119'),
 			'diagnostics' => null,
 			'dump_possible_options' => null,
+			'debug_render_test' => null,
 			);
 
 		$individual_times = array();
@@ -54,7 +57,7 @@ class debug_self_test implements pts_option_interface
 				$c_start = microtime(true);
 				pts_client::execute_command($command, $args);
 				$c_finish = microtime(true);
-				array_push($individual_times[$command], ($c_finish - $c_start));
+				$individual_times[$command][] = ($c_finish - $c_start);
 			}
 		}
 		$finish = microtime(true);
@@ -68,9 +71,9 @@ class debug_self_test implements pts_option_interface
 			echo strtoupper($component) . ': ' . (str_repeat(' ', $longest_c - strlen($component))) . pts_math::set_precision(round(array_sum($times) / count($times), 3), 3) . ' seconds' . PHP_EOL;
 		}
 
-		echo PHP_EOL . 'ELAPSED TIME: ' . (str_repeat(' ', $longest_c - strlen('ELAPSED TIME'))) . round($finish - $start, 3) . ' seconds';
+		echo PHP_EOL . 'TOTAL ELAPSED TIME: ' . (str_repeat(' ', $longest_c - strlen('ELAPSED TIME'))) . round($finish - $start, 3) . ' seconds';
 		echo PHP_EOL . 'PEAK MEMORY USAGE: ' . (str_repeat(' ', $longest_c - strlen('PEAK MEMORY USAGE'))) . round(memory_get_peak_usage(true) / 1048576, 3) . ' MB';
-		echo PHP_EOL . 'PEAK MEMORY USAGE (emalloc): ' . (str_repeat(' ', $longest_c - strlen('PEAK MEMORY USAGE (emalloc)'))) . round(memory_get_peak_usage() / 1048576, 3) . ' MB';
+		echo PHP_EOL . 'PEAK MEMORY USAGE EMALLOC: ' . (str_repeat(' ', $longest_c - strlen('PEAK MEMORY USAGE (emalloc)'))) . round(memory_get_peak_usage() / 1048576, 3) . ' MB';
 		echo PHP_EOL;
 	}
 }

pts-core/commands/debug_test_download_links.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2010, Phoronix Media
-	Copyright (C) 2010, Michael Larabel
+	Copyright (C) 2010 - 2018, Phoronix Media
+	Copyright (C) 2010 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -28,7 +28,7 @@ class debug_test_download_links implements pts_option_interface
 	public static function argument_checks()
 	{
 		return array(
-		new pts_argument_check(0, array('pts_types', 'is_test_or_suite'), null)
+		new pts_argument_check(0, array('pts_types', 'identifier_to_object'), null)
 		);
 	}
 	public static function run($r)
@@ -37,7 +37,7 @@ class debug_test_download_links implements pts_option_interface
 		{
 			echo 'Checking: ' . $test_profile . PHP_EOL;
 
-			foreach(pts_test_install_request::read_download_object_list($test_profile) as $test_file_download)
+			foreach($test_profile->get_downloads() as $test_file_download)
 			{
 				foreach($test_file_download->get_download_url_array() as $url)
 				{
@@ -48,7 +48,7 @@ class debug_test_download_links implements pts_option_interface
 
 					if($file_pointer == false)
 					{
-						echo PHP_EOL . 'DOWNLOAD: ' . $test_file_download->get_filename() . ' / ' . $url . PHP_EOL;
+						echo PHP_EOL . 'BAD URL: ' . $test_file_download->get_filename() . ' / ' . $url . PHP_EOL;
 					}
 					else
 					{

pts-core/commands/debug_websocket_client.php

@@ -42,22 +42,22 @@ class debug_websocket_client implements pts_option_interface
 		pts_web_socket_client::$debug_mode = true;
 		$ws_client = new pts_web_socket_client($web_socket_address, $web_socket_port);
 
-		$ws_client->send_message('TEST 123');
-		$ws_client->send_message('BIG TEST SENDDDDDDDDDDDDDDDDDDDDDDDDDDDDD MESSSSSSSSSSSSSSSSSSSSSSSSSAGE');
-		$ws_client->send_message('1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE END END END 321');
-		$ws_client->send_message('TEST MESSAGE');
+		$ws_client->send('TEST 123');
+		$ws_client->send('BIG TEST SENDDDDDDDDDDDDDDDDDDDDDDDDDDDDD MESSSSSSSSSSSSSSSSSSSSSSSSSAGE');
+		$ws_client->send('1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE 1234678910 VERY LONG MESSAGE END END END 321');
+		$ws_client->send('TEST MESSAGE');
 
 		while(1)
 		{
-			$msg = $ws_client->receive_message();
+			$msg = $ws_client->receive();
 			if($msg != null)
 			{
-				echo PHP_EOL . 'RECEIVED: ' . $ws_client->receive_message() . PHP_EOL;
+				echo PHP_EOL . 'RECEIVED: ' . $ws_client->receive() . PHP_EOL;
 			}
 
 			//if(date('s') % 5 == 0)
 			//{
-			//	$ws_client->send_message(rand(1, getrandmax()));
+			//	$ws_client->send(rand(1, getrandmax()));
 			//}
 		}
 		$ws_client->disconnect();

pts-core/commands/default_benchmark.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2010, Phoronix Media
-	Copyright (C) 2009 - 2010, Michael Larabel
+	Copyright (C) 2009 - 2015, Phoronix Media
+	Copyright (C) 2009 - 2015, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -34,7 +34,8 @@ class default_benchmark implements pts_option_interface
 	public static function run($r)
 	{
 		pts_test_installer::standard_install($r);
-		pts_test_run_manager::standard_run($r, pts_c::defaults_mode);
+		$test_run_manager = new pts_test_run_manager(false, 2);
+		$test_run_manager->standard_run($r);
 	}
 }
 

pts-core/commands/default_run.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2010, Phoronix Media
-	Copyright (C) 2009 - 2010, Michael Larabel
+	Copyright (C) 2009 - 2015, Phoronix Media
+	Copyright (C) 2009 - 2015, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -33,7 +33,8 @@ class default_run implements pts_option_interface
 	}
 	public static function run($r)
 	{
-		pts_test_run_manager::standard_run($r, pts_c::defaults_mode);
+		$test_run_manager = new pts_test_run_manager(false, 2);
+		$test_run_manager->standard_run($r);
 	}
 }
 

pts-core/commands/detailed_system_info.php

@@ -1,57 +0,0 @@
-<?php
-
-/*
-	Phoronix Test Suite
-	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2011 - 2012, Phoronix Media
-	Copyright (C) 2011 - 2012, Michael Larabel
-
-	This program is free software; you can redistribute it and/or modify
-	it under the terms of the GNU General Public License as published by
-	the Free Software Foundation; either version 3 of the License, or
-	(at your option) any later version.
-
-	This program is distributed in the hope that it will be useful,
-	but WITHOUT ANY WARRANTY; without even the implied warranty of
-	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-	GNU General Public License for more details.
-
-	You should have received a copy of the GNU General Public License
-	along with this program. If not, see <http://www.gnu.org/licenses/>.
-*/
-
-class detailed_system_info implements pts_option_interface
-{
-	const doc_section = 'System';
-	const doc_description = 'Display detailed information about the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.';
-
-	public static function run($r)
-	{
-		pts_client::$display->generic_heading('System Information');
-		echo 'Hardware:' . PHP_EOL . phodevi::system_hardware(true) . PHP_EOL . PHP_EOL;
-		echo 'Software:' . PHP_EOL . phodevi::system_software(true) . PHP_EOL . PHP_EOL;
-
-		//
-		// Processor Information
-		//
-
-		$cpu_flags = phodevi_cpu::get_cpu_flags();
-		echo PHP_EOL . 'PROCESSOR:' . PHP_EOL . PHP_EOL;
-		echo 'Core Count: ' . phodevi_cpu::cpuinfo_core_count() . PHP_EOL;
-		echo 'Thread Count: ' . phodevi_cpu::cpuinfo_thread_count() . PHP_EOL;
-		echo 'Cache Size: ' . phodevi_cpu::cpuinfo_cache_size() . ' KB' . PHP_EOL;
-
-		echo 'Instruction Set Extensions: ' . phodevi_cpu::instruction_set_extensions() . PHP_EOL;
-		echo 'AES Encryption: ' . ($cpu_flags & phodevi_cpu::get_cpu_feature_constant('aes') ? 'YES' : 'NO') . PHP_EOL;
-		echo 'Energy Performance Bias: ' . ($cpu_flags & phodevi_cpu::get_cpu_feature_constant('epb') ? 'YES' : 'NO') . PHP_EOL;
-		echo 'Virtualization: ' . (phodevi_cpu::virtualization_technology() ? phodevi_cpu::virtualization_technology() : 'NO') . PHP_EOL;
-
-		// Other info
-		foreach(pts_test_run_manager::pull_test_notes(true) as $test_note_head => $test_note)
-		{
-			echo ucwords(str_replace('-', ' ', $test_note_head)) . ': ' . $test_note . PHP_EOL;
-		}
-	}
-}
-
-?>

pts-core/commands/diagnostics.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2013, Phoronix Media
-	Copyright (C) 2008 - 2013, Michael Larabel
+	Copyright (C) 2008 - 2018, Phoronix Media
+	Copyright (C) 2008 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -29,18 +29,23 @@ class diagnostics implements pts_option_interface
 	{
 		foreach(pts_define(-1) as $constant => $constant_value)
 		{
-			echo $constant . ' = ' . $constant_value . PHP_EOL;
+			if(strpos($constant, 'GSID') !== false || strpos($constant, '_ID') !== false)
+			{
+				continue;
+			}
+
+			echo pts_client::cli_just_bold($constant) . ' = ' . $constant_value . PHP_EOL;
 		}
 
 		echo PHP_EOL . 'Variables That Can Be Used As Result Identifiers / File Names:' . PHP_EOL;
 		foreach(pts_client::user_run_save_variables() as $var => $var_value)
 		{
-			echo $var . ' = ' . $var_value . PHP_EOL;
+			echo pts_client::cli_just_bold($var) . ' = ' . $var_value . PHP_EOL;
 		}
 		echo PHP_EOL . 'Environmental Variables (accessible via test scripts):' . PHP_EOL;
 		foreach(pts_client::environmental_variables() as $var => $var_value)
 		{
-			echo $var . ' = ' . $var_value . PHP_EOL;
+			echo pts_client::cli_just_bold($var) . ' = ' . $var_value . PHP_EOL;
 		}
 		echo PHP_EOL;
 	}

pts-core/commands/download_test_files.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2011 - 2013, Phoronix Media
-	Copyright (C) 2011 - 2013, Michael Larabel
+	Copyright (C) 2011 - 2014, Phoronix Media
+	Copyright (C) 2011 - 2014, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by

pts-core/commands/dry_run.php

@@ -0,0 +1,51 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2013 - 2018, Phoronix Media
+	Copyright (C) 2013 - 2018, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class dry_run implements pts_option_interface
+{
+	const doc_section = 'Batch Testing';
+	const doc_description = 'This option and its arguments pre-set the Phoronix Test Suite batch run mode with enforcing of defaults to not save any results and other behavior intended for a dry/test run. This option is primarily intended for testing/evaluation purposes.';
+
+	public static function argument_checks()
+	{
+		return array(
+		new pts_argument_check('VARIABLE_LENGTH', array('pts_types', 'identifier_to_object'), null)
+		);
+	}
+	public static function run($r)
+	{
+		pts_test_installer::standard_install($r);
+		$test_run_manager = new pts_test_run_manager(array(
+			'UploadResults' => false,
+			'SaveResults' => false,
+			'PromptForTestDescription' => false,
+			'RunAllTestCombinations' => false,
+			'PromptSaveName' => false,
+			'PromptForTestIdentifier' => false,
+			'OpenBrowser' => false
+			));
+
+		$test_run_manager->standard_run($r);
+	}
+}
+
+?>

pts-core/commands/dump_core_storage.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009, Phoronix Media
-	Copyright (C) 2009, Michael Larabel
+	Copyright (C) 2009 - 2014, Phoronix Media
+	Copyright (C) 2009 - 2014, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -29,28 +29,31 @@ class dump_core_storage implements pts_option_interface
 
 		foreach($pso->get_objects() as $pso_index => $pso_object)
 		{
-			if($pso_index != 'phodevi_smart_cache')
+			if(!in_array($pso_index, array('global_reported_hw', 'global_reported_sw', 'global_reported_usb', 'global_reported_pci', 'phodevi_smart_cache')))
 			{
-				echo $pso_index . ': ';
-
-				if(is_array($pso_object))
-				{
-					foreach($pso_object as $key => $element)
-					{
-						echo PHP_EOL . "\t" . $key . ': ' . $element;
-					}
-				}
-				else
-				{
-					echo $pso_object;
-				}
-
-				echo PHP_EOL;
+				self::print_element($pso_index, $pso_object, 0);
 			}
 		}
 
 		echo PHP_EOL;
 	}
+	private static function print_element($in, $el, $depth)
+	{
+		//echo $in . ': ';
+
+		if(is_array($el))
+		{
+			foreach($el as $key => $element)
+			{
+				echo PHP_EOL . str_repeat("\t", $depth) . $in . ': ';
+				self::print_element($key, $element, ($depth + 1));
+			}
+		}
+		else
+		{
+			echo PHP_EOL . str_repeat("\t", $depth) . $in . ': ' . $el;
+		}
+	}
 }
 
 ?>

pts-core/commands/dump_documentation.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2010 - 2014, Phoronix Media
-	Copyright (C) 2010 - 2014, Michael Larabel
+	Copyright (C) 2010 - 2018, Phoronix Media
+	Copyright (C) 2010 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -24,25 +24,15 @@ class dump_documentation implements pts_option_interface
 {
 	public static function run($r)
 	{
-		if(is_file('/usr/share/php/fpdf/fpdf.php'))
-		{
-			include_once('/usr/share/php/fpdf/fpdf.php');
-		}
-		else
-		{
-			echo PHP_EOL . 'The FPDF library must be installed.' . PHP_EOL . PHP_EOL;
-			return;
-		}
-
-		$pdf = new pts_pdf_template(pts_title(false), 'Test Client Documentation');
-		$html_doc = new pts_html_template(pts_title(false), 'Test Client Documentation');
+		$pdf = new pts_pdf_template(pts_core::program_title(false), 'Test Client Documentation');
+		$html_doc = new pts_html_template(pts_core::program_title(false), 'Test Client Documentation');
 
 		$pdf->AddPage();
 		$pdf->Image(PTS_CORE_STATIC_PATH . 'images/pts-308x160.png', 69, 85, 73, 38, 'PNG', 'http://www.phoronix-test-suite.com/');
 		$pdf->Ln(120);
 		$pdf->WriteStatement('www.phoronix-test-suite.com', 'C', 'http://www.phoronix-test-suite.com/');
 		$pdf->Ln(15);
-		$pdf->WriteBigHeaderCenter(pts_title(true));
+		$pdf->WriteBigHeaderCenter(pts_core::program_title(true));
 		$pdf->WriteHeaderCenter('User Manual');
 		//$pdf->WriteText($result_file->get_description());
 
@@ -50,6 +40,7 @@ class dump_documentation implements pts_option_interface
 
 		// Write the test options HTML
 		$dom = new DOMDocument();
+		$dom->formatOutput = true;
 		$html = $dom->createElement('html');
 		$dom->appendChild($html);
 		$head = $dom->createElement('head');
@@ -103,7 +94,7 @@ class dump_documentation implements pts_option_interface
 		$body = $dom->createElement('body');
 		$html->appendChild($body);
 
-		$p = $dom->createElement('p', 'The following list is the modules included with the Phoronix Test Suite that are intended to extend the functionality of pts-core. Some of these options have commands that can be run directly in a similiar manner to the other Phoronix Test Suite user commands. Some modules are just meant to be loaded directly by adding the module name to the LoadModules tag in ~/.phoronix-test-suite/user-config.xml or via the PTS_MODULES environmental variable. A list of available modules is also available by running ');
+		$p = $dom->createElement('p', 'The following list is the modules included with the Phoronix Test Suite that are intended to extend the functionality of pts-core. Some of these options have commands that can be run directly in a similiar manner to the other Phoronix Test Suite user commands. Some modules are just meant to be loaded directly by adding the module name to the LoadModules tag in ~/.phoronix-test-suite/user-config.xml or via the PTS_MODULES environment variable. A list of available modules is also available by running ');
 		$em = $dom->createElement('em', 'phoronix-test-suite list-modules.');
 		$p->appendChild($em);
 		$phr = $dom->createElement('hr');
@@ -215,10 +206,12 @@ class dump_documentation implements pts_option_interface
 		$dom->saveHTMLFile(PTS_PATH . 'documentation/stubs/55_virtual_suites.html');
 
 		// Load the HTML documentation
+		$md = new pts_md_template();
 		foreach(pts_file_io::glob(PTS_PATH . 'documentation/stubs/*_*.html') as $html_file)
 		{
 			$pdf->html_to_pdf($html_file);
 			$html_doc->html_to_html($html_file);
+			$md->html_to_md($html_file);
 		}
 
 		if(!is_writable(PTS_PATH . 'documentation/'))
@@ -232,6 +225,8 @@ class dump_documentation implements pts_option_interface
 			$html_doc->Output(PTS_PATH . 'documentation/phoronix-test-suite.html');
 			echo PHP_EOL . 'Saved To: ' . $pdf_file . PHP_EOL . PHP_EOL;
 
+			$md->Output(PTS_PATH . 'documentation/phoronix-test-suite.md');
+
 			// Also re-generate the man page
 			$man_page = '.TH phoronix-test-suite 1  "www.phoronix-test-suite.com" "' . PTS_VERSION . '"' . PHP_EOL . '.SH NAME' . PHP_EOL;
 			$man_page .= 'phoronix-test-suite \- The Phoronix Test Suite is an extensible open-source platform for performing testing and performance evaluation.' . PHP_EOL;
@@ -263,11 +258,39 @@ class dump_documentation implements pts_option_interface
 
 
 		// simple README
-		$readme = 'Phoronix Test Suite ' . PTS_VERSION . PHP_EOL . 'http://www.phoronix-test-suite.com/' . PHP_EOL . '#####################################' . PHP_EOL . PHP_EOL;
+		$readme = '# Phoronix Test Suite ' . PTS_VERSION . PHP_EOL . 'http://www.phoronix-test-suite.com/' . PHP_EOL . PHP_EOL;
 		$readme .= pts_documentation::basic_description() . PHP_EOL . PHP_EOL;
 		$readme .= pts_file_io::file_get_contents(PTS_PATH . 'documentation/stubs/readme-basics.txt') . PHP_EOL . PHP_EOL;
 		$readme = wordwrap($readme, 80, PHP_EOL);
-		file_put_contents(PTS_PATH . 'README', $readme);
+		file_put_contents(PTS_PATH . 'README.md', $readme);
+
+		// Phoromatic Documentation
+		$pdf = new pts_pdf_template(pts_core::program_title(false), 'Phoromatic Documentation');
+		$html_doc = new pts_html_template(pts_core::program_title(false), 'Phoromatic Documentation');
+
+		$pdf->AddPage();
+		$pdf->Image(PTS_CORE_STATIC_PATH . 'images/pts-308x160.png', 69, 85, 73, 38, 'PNG', 'http://www.phoronix-test-suite.com/');
+		$pdf->Ln(120);
+		$pdf->WriteStatement('www.phoronix-test-suite.com', 'C', 'http://www.phoronix-test-suite.com/');
+		$pdf->Ln(15);
+		$pdf->Image(PTS_CORE_STATIC_PATH . 'images/phoromatic-390x56.png', 55, 250, 0, 0, 'PNG', 'http://www.phoronix-test-suite.com/');
+		//$pdf->Image(PTS_CORE_STATIC_PATH . 'images/phoromatic-390x56.png', 69, 85, 73, 38, 'PNG', 'http://www.phoromatic.com/');
+		$pdf->WriteBigHeaderCenter(pts_core::program_title(true));
+		$pdf->WriteHeaderCenter('Phoromatic User Manual');
+		$pdf->html_to_pdf(PTS_PATH . 'documentation/phoromatic.html');
+		$pdf_file = PTS_PATH . 'documentation/phoromatic.pdf';
+		$pdf->Output($pdf_file);
+
+		echo PHP_EOL . 'Saved To: ' . $pdf_file . PHP_EOL . PHP_EOL;
+
+		$md = new pts_md_template();
+		$md->html_to_md(PTS_PATH . 'documentation/phoromatic.html');
+		$md->Output(PTS_PATH . 'documentation/phoromatic.md');
+
+		$md = new pts_md_template();
+		$md->html_to_md(PTS_PATH . 'documentation/phoronix-test-suite-windows.html');
+		$md->Output(PTS_PATH . 'documentation/phoronix-test-suite-window.md');
+
 	}
 }
 

pts-core/commands/dump_file_info.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2011, Phoronix Media
-	Copyright (C) 2009 - 2011, Michael Larabel
+	Copyright (C) 2017, Phoronix Media
+	Copyright (C) 2017, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -20,37 +20,30 @@
 	along with this program. If not, see <http://www.gnu.org/licenses/>.
 */
 
-class sys_v5 implements phodevi_sensor
+class dump_file_info implements pts_option_interface
 {
-	public static function get_type()
+	public static function run($r)
 	{
-		return 'sys';
-	}
-	public static function get_sensor()
-	{
-		return 'v5-voltage';
-	}
-	public static function get_unit()
-	{
-		return 'Volts';
-	}
-	public static function support_check()
-	{
-		$test = self::read_sensor();
-		return is_numeric($test) && $test != -1;
-	}
-	public static function read_sensor()
-	{
-		if(phodevi::is_linux())
+		pts_client::$display->generic_heading('File Information');
+		if(empty($r))
 		{
-			$sensor = phodevi_linux_parser::read_sensors(array('V5', '+5V'));
+			echo PHP_EOL . 'No files passed.' . PHP_EOL;
 		}
-		else
+		foreach($r as $f)
 		{
-			$sensor = -1;
-		}
+			if(!is_file($f))
+			{
+				echo PHP_EOL . 'Not a file: ' . $f . PHP_EOL;
+			}
+			else
+			{
+				echo PHP_EOL . 'FILE:   ' . basename($f) . PHP_EOL;
+				echo 'MD5:    ' . md5_file($f) . PHP_EOL;
+				echo 'SHA256: ' . hash_file('sha256', $f) . PHP_EOL;
+				echo 'SIZE:   ' . filesize($f) . PHP_EOL;
+			}
 
-		return $sensor;
+		}
 	}
 }
 

pts-core/commands/dump_phodevi_properties.php

@@ -0,0 +1,53 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2018, Phoronix Media
+	Copyright (C) 2018, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class dump_phodevi_properties implements pts_option_interface
+{
+	public static function run($r)
+	{
+		$properties = phodevi::read_all_properties();
+
+		foreach($properties as $component => $component_properties)
+		{
+			echo strtoupper($component) . PHP_EOL;
+			foreach($component_properties as $property => $value)
+			{
+				echo '     ' . $property . ' = ';
+
+				if(is_array($value))
+				{
+					var_dump($value);
+/*					echo PHP_EOL;
+					foreach($value as $i => $j)
+						echo '         ' . $i . ' = ' . $j . PHP_EOL; */
+				}
+				else
+				{
+					echo $value . PHP_EOL;
+				}
+			}
+			echo PHP_EOL;
+		}
+	}
+}
+
+?>

pts-core/commands/dump_possible_options.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2010, Phoronix Media
-	Copyright (C) 2009 - 2010, Michael Larabel
+	Copyright (C) 2009 - 2018, Phoronix Media
+	Copyright (C) 2009 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -24,20 +24,8 @@ class dump_possible_options implements pts_option_interface
 {
 	public static function run($r)
 	{
-		$options = array();
-
-		foreach(pts_file_io::glob(PTS_COMMAND_PATH . '*.php') as $option_php)
-		{
-			$name = str_replace('_', '-', basename($option_php, '.php'));
-
-			if(!in_array(pts_strings::first_in_string($name, '-'), array('dump', 'debug', 'task')))
-			{
-				array_push($options, $name);
-			}
-		}
-
 		$is_true = isset($r[0]) && $r[0] == 'TRUE';
-		echo implode($is_true ? ' ' : PHP_EOL, $options) . ($is_true ? null : PHP_EOL);
+		echo implode($is_true ? ' ' : PHP_EOL, pts_client::possible_sub_commands()) . ($is_true ? null : PHP_EOL);
 	}
 }
 

pts-core/commands/dump_suites_to_git.php

@@ -0,0 +1,80 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2017, Phoronix Media
+	Copyright (C) 2017, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class dump_suites_to_git implements pts_option_interface
+{
+	const doc_section = 'OpenBenchmarking.org';
+	const doc_description = 'This option will create a Git repository of OpenBenchmarking.org test suites.';
+
+	public static function run($r)
+	{
+		pts_client::$display->generic_heading('OpenBenchmarking.org Tests To Suites');
+		$path_to_git = getenv('TEST_SUITES_GIT_PATH') . '/';
+		if(!pts_client::executable_in_path('git'))
+		{
+			echo PHP_EOL . 'git was not found on the system.' . PHP_EOL . PHP_EOL;
+			return false;
+		}
+		if(empty($path_to_git) || !is_dir($path_to_git) || !is_writable($path_to_git))
+		{
+			echo PHP_EOL . 'TEST_SUITES_GIT_PATH must be set or the set directory is not writable/present.' . PHP_EOL . PHP_EOL;
+			return false;
+		}
+
+		shell_exec('cd ' . $path_to_git . ' && git pull');
+
+		foreach(pts_openbenchmarking::linked_repositories() as $repo)
+		{
+			if($repo == 'local')
+			{
+				// Skip local since it's a fake repository
+				continue;
+			}
+			if(!is_dir($path_to_git . $repo))
+			{
+				pts_file_io::mkdir($path_to_git . $repo);
+			}
+			$repo_index = pts_openbenchmarking::read_repository_index($repo);
+			$changes = pts_openbenchmarking_client::fetch_repository_changelog_full($repo);
+
+			foreach($changes['suites'] as $suite_identifier => $d)
+			{
+				foreach($d['changes'] as $suite_version => $dd)
+				{
+					$suite = $repo . '/' .  $suite_identifier . '-' . $suite_version; //  . ' ' . $dd['commit_description'] . ' ' . date('d F Y', $dd['last_updated']) . PHP_EOL;
+					if(is_dir($path_to_git . $suite))
+						continue;
+
+					pts_openbenchmarking::download_test_suite($suite, $path_to_git);
+					if(is_dir($path_to_git . $suite))
+					{
+						$test_suite = new pts_test_suite($suite);
+						echo 'git commit -m "' . $suite . ': ' . $dd['commit_description'] . '" --author="' . $test_suite->get_maintainer() . ' <no-reply@openbenchmarking.org>" --date="' . date(DATE_RFC2822, $dd['last_updated']) . '" ' . $suite . PHP_EOL;
+						shell_exec('cd ' . $path_to_git . ' && git add ' . $suite . ' && git commit -m "' . $suite . ': ' . $dd['commit_description'] . '" --author="' . $test_suite->get_maintainer() . ' <no-reply@openbenchmarking.org>" --date="' . date(DATE_RFC2822, $dd['last_updated']) . '" ' . $suite);
+					}
+				}
+			}
+		}
+	}
+}
+
+?>

pts-core/commands/dump_tests_to_git.php

@@ -0,0 +1,81 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2017, Phoronix Media
+	Copyright (C) 2017, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class dump_tests_to_git implements pts_option_interface
+{
+	const doc_section = 'OpenBenchmarking.org';
+	const doc_description = 'This option will create a Git repository of OpenBenchmarking.org test profiles.';
+
+	public static function run($r)
+	{
+		pts_client::$display->generic_heading('OpenBenchmarking.org Tests To Git');
+		$path_to_git = getenv('TEST_PROFILES_GIT_PATH') . '/';
+		if(!pts_client::executable_in_path('git'))
+		{
+			echo PHP_EOL . 'git was not found on the system.' . PHP_EOL . PHP_EOL;
+			return false;
+		}
+		if(empty($path_to_git) || !is_dir($path_to_git) || !is_writable($path_to_git))
+		{
+			echo PHP_EOL . 'TEST_PROFILES_GIT_PATH must be set or the set directory is not writable/present.' . PHP_EOL . PHP_EOL;
+			return false;
+		}
+
+		shell_exec('cd ' . $path_to_git . ' && git pull');
+
+		foreach(pts_openbenchmarking::linked_repositories() as $repo)
+		{
+			if($repo == 'local')
+			{
+				// Skip local since it's a fake repository
+				continue;
+			}
+			if(!is_dir($path_to_git . $repo))
+			{
+				pts_file_io::mkdir($path_to_git . $repo);
+			}
+			$repo_index = pts_openbenchmarking::read_repository_index($repo);
+			$changes = pts_openbenchmarking_client::fetch_repository_changelog_full($repo);
+
+			foreach($changes['tests'] as $test_identifier => $d)
+			{
+				foreach($d['changes'] as $test_version => $dd)
+				{
+					$test = $repo . '/' .  $test_identifier . '-' . $test_version; //  . ' ' . $dd['commit_description'] . ' ' . date('d F Y', $dd['last_updated']) . PHP_EOL;
+					if(is_dir($path_to_git . $test))
+						continue;
+
+					pts_openbenchmarking::download_test_profile($test, $path_to_git);
+					if(is_dir($path_to_git . $test))
+					{
+						$test_profile = new pts_test_profile($test);
+						$maintainer = $test_profile->get_maintainer() != null ? $test_profile->get_maintainer() : 'Phoronix Test Suite';
+						echo 'git commit -m "' . $test . ': ' . $dd['commit_description'] . '" --author="' . $maintainer . ' <no-reply@openbenchmarking.org>" --date="' . date(DATE_RFC2822, $dd['last_updated']) . '" ' . $test . PHP_EOL;
+						shell_exec('cd ' . $path_to_git . ' && git add ' . $test . ' && git commit -m "' . $test . ': ' . $dd['commit_description'] . '" --author="' . $maintainer . ' <no-reply@openbenchmarking.org>" --date="' . date(DATE_RFC2822, $dd['last_updated']) . '" ' . $test);
+					}
+				}
+			}
+		}
+	}
+}
+
+?>

pts-core/commands/edit_result_file.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2014, Phoronix Media
-	Copyright (C) 2014, Michael Larabel
+	Copyright (C) 2014 - 2018, Phoronix Media
+	Copyright (C) 2014 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -31,37 +31,24 @@ class edit_result_file implements pts_option_interface
 		new pts_argument_check(0, array('pts_types', 'is_result_file'), null)
 		);
 	}
-	public static function invalid_command($passed_args = null)
-	{
-		pts_tests::recently_saved_results();
-	}
 	public static function run($r)
 	{
 		$result_file = new pts_result_file($r[0]);
-
 		$result_title = $result_file->get_title();
 		echo PHP_EOL . 'Current Result Title: ' . $result_title . PHP_EOL;
 		$new_title = pts_user_io::prompt_user_input('Enter New Title');
-
 		if(!empty($new_title))
 		{
-			$result_title = $new_title;
+			$result_file->set_title($new_title);
 		}
-
 		$result_description = $result_file->get_description();
 		echo PHP_EOL . 'Current Result Description: ' . $result_description . PHP_EOL;
 		$new_description = pts_user_io::prompt_user_input('Enter New Description');
-
 		if(!empty($new_description))
 		{
-			$result_description = $new_description;
+			$result_file->set_description($new_description);
 		}
-
-		$result_file_writer = new pts_result_file_writer();
-		$result_file_writer->add_result_file_meta_data($result_file, null, $new_title, $new_description);
-		$result_file_writer->add_system_information_from_result_file($result_file);
-		$result_file_writer->add_results_from_result_file($result_file);
-		pts_client::save_test_result($r[0] . '/composite.xml', $result_file_writer->get_xml());
+		pts_client::save_test_result($result_file->get_file_location(), $result_file->get_xml());
 		pts_client::display_web_page(PTS_SAVE_RESULTS_PATH . $r[0] . '/index.html');
 	}
 }

pts-core/commands/enterprise_setup.php

@@ -0,0 +1,49 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2014 - 2017, Phoronix Media
+	Copyright (C) 2014 - 2017, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class enterprise_setup implements pts_option_interface
+{
+	const doc_section = 'User Configuration';
+	const doc_description = 'This option can be run by enterprise users immediately after package installation or as part of an in-house setup script. Running this command will ensure the phoronix-test-suite program is never interrupted on new runs to accept user agreement changes and defaults the anonymous usage reporting to being disabled and other conservative defaults.';
+
+	public static function run($r)
+	{
+		$force_options = array(
+			'PhoronixTestSuite/Options/OpenBenchmarking/AnonymousUsageReporting' => 'FALSE'
+			);
+
+		if(pts_network::internet_support_available() == false)
+		{
+			$force_options['PhoronixTestSuite/Options/Networking/NoInternetCommunication'] = 'TRUE';
+		}
+
+		pts_config::user_config_generate($force_options);
+
+		$pso = pts_storage_object::recover_from_file(PTS_CORE_STORAGE);
+		$pso->add_object('user_agreement_cs', 'enterprise-agree');
+		$pso->save_to_file(PTS_CORE_STORAGE);
+
+		echo PHP_EOL . 'Enterprise setup tasks executed.' . PHP_EOL . PHP_EOL;
+	}
+}
+
+?>

pts-core/commands/estimate_run_time.php

@@ -0,0 +1,63 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2017, Phoronix Media
+	Copyright (C) 2017, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class estimate_run_time implements pts_option_interface
+{
+	const doc_section = 'Testing';
+	const doc_description = 'This option will provide estimates for test run-time / length.';
+
+	public static function argument_checks()
+	{
+		return array(
+		new pts_argument_check(0, array('pts_types', 'identifier_to_object'))
+		);
+	}
+	public static function run($args)
+	{
+		echo PHP_EOL;
+
+		if($args[0] == 'pts/all' || empty($args))
+		{
+			$args = pts_openbenchmarking::available_tests(false);
+		}
+
+		$tests = array();
+		$total_time = 0;
+		$test_count = 0;
+		foreach($args as $arg)
+		{
+			foreach(pts_types::identifiers_to_test_profile_objects($arg) as $t)
+			{
+				$tests[] = array($t->get_identifier(), pts_strings::format_time($t->get_estimated_run_time()));
+				$total_time += $t->get_estimated_run_time();
+				$test_count++;
+			}
+		}
+		if($test_count > 1 && $total_time > 0)
+		{
+			echo pts_user_io::display_text_table($tests);
+			echo PHP_EOL . PHP_EOL . 'TOTAL TIME ESTIMATE: ' . pts_strings::format_time($total_time) . PHP_EOL . PHP_EOL;
+		}
+	}
+}
+
+?>

pts-core/commands/extract_from_result_file.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2013, Phoronix Media
-	Copyright (C) 2009 - 2013, Michael Larabel
+	Copyright (C) 2009 - 2015, Phoronix Media
+	Copyright (C) 2009 - 2015, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -33,9 +33,7 @@ class extract_from_result_file implements pts_option_interface
 	}
 	public static function run($args)
 	{
-		$result = $args[0];
-
-		$result_file = new pts_result_file($result);
+		$result_file = new pts_result_file($args[0]);
 		$result_file_identifiers = $result_file->get_system_identifiers();
 
 		if(count($result_file_identifiers) < 2)
@@ -45,12 +43,8 @@ class extract_from_result_file implements pts_option_interface
 		}
 
 		$extract_identifiers = pts_strings::comma_explode(pts_user_io::prompt_text_menu('Select the test run(s) to extract', $result_file_identifiers, true));
-		$extract_selects = array();
-
-		foreach($extract_identifiers as $extract_identifier)
-		{
-			array_push($extract_selects, new pts_result_merge_select($result, $extract_identifier));
-		}
+		$remove_identifiers = array_diff($result_file_identifiers, $extract_identifiers);
+		$result_file->remove_run($remove_identifiers);
 
 		do
 		{
@@ -58,10 +52,9 @@ class extract_from_result_file implements pts_option_interface
 			$extract_to = pts_user_io::read_user_input();
 			$extract_to = pts_test_run_manager::clean_save_name($extract_to);
 		}
-		while(empty($extract_to) || pts_result_file::is_test_result_file($extract_to));
+		while(empty($extract_to));
 
-		$extract_result = call_user_func_array(array('pts_merge', 'merge_test_results'), $extract_selects);
-		pts_client::save_test_result($extract_to . '/composite.xml', $extract_result);
+		pts_client::save_test_result($extract_to . '/composite.xml', $result_file->get_xml());
 		pts_client::display_web_page(PTS_SAVE_RESULTS_PATH . $extract_to . '/index.html');
 	}
 }

pts-core/commands/finish_run.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2011, Phoronix Media
-	Copyright (C) 2009 - 2011, Michael Larabel
+	Copyright (C) 2009 - 2018, Phoronix Media
+	Copyright (C) 2009 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -39,9 +39,14 @@ class finish_run implements pts_option_interface
 		$system_identifiers = $result_file->get_system_identifiers();
 		$test_positions = array();
 
-		$pos = 0;
-		foreach($result_file->get_result_objects() as $result_object)
+		foreach($result_file->get_result_objects() as $pos => $result_object)
 		{
+			// Don't load null test profile identifier tests into the run queue
+			if($result_object->test_profile->get_identifier() == null)
+			{
+				continue;
+			}
+
 			$this_result_object_identifiers = $result_object->test_result_buffer->get_identifiers();
 
 			foreach($system_identifiers as $system_identifier)
@@ -53,13 +58,10 @@ class finish_run implements pts_option_interface
 						$test_positions[$system_identifier] = array();
 					}
 
-					array_push($test_positions[$system_identifier], $pos);
+					$test_positions[$system_identifier][] = $pos;
 				}
 			}
-
-			$pos++;
 		}
-
 		$incomplete_identifiers = array_keys($test_positions);
 
 		if(count($incomplete_identifiers) == 0)
@@ -69,15 +71,14 @@ class finish_run implements pts_option_interface
 		}
 
 		$selected = pts_user_io::prompt_text_menu('Select which incomplete test run you would like to finish', $incomplete_identifiers);
+		$test_run_manager = new pts_test_run_manager();
 
 		// Now run it
-		if(pts_test_run_manager::initial_checks($args[0]) == false)
+		if($test_run_manager->initial_checks($args[0]) == false)
 		{
 			return false;
 		}
 
-		$test_run_manager = new pts_test_run_manager(pts_c::is_recovering);
-
 		// Load the tests to run
 		if($test_run_manager->load_result_file_to_run($save_name, $selected, $result_file, $test_positions[$selected]) == false)
 		{

pts-core/commands/force_install.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2014, Phoronix Media
-	Copyright (C) 2009 - 2014, Michael Larabel
+	Copyright (C) 2009 - 2018, Phoronix Media
+	Copyright (C) 2009 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -22,9 +22,13 @@
 
 class force_install implements pts_option_interface
 {
-	const doc_section = 'Asset Creation';
-	const doc_description = 'This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem.';
+	const doc_section = 'Test Installation';
+	const doc_description = 'This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem or wishing to re-install test(s) due to compiler or other environmental changes.';
 
+	public static function command_aliases()
+	{
+		return array('reinstall', 're-install');
+	}
 	public static function argument_checks()
 	{
 		return array(
@@ -33,11 +37,7 @@ class force_install implements pts_option_interface
 	}
 	public static function run($r)
 	{
-		pts_test_installer::standard_install($r, pts_c::force_install);
-	}
-	public static function invalid_command($passed_args = null)
-	{
-		pts_tests::invalid_command_helper($passed_args);
+		pts_test_installer::standard_install($r, true);
 	}
 }
 

pts-core/commands/gui.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2012 - 2014, Phoronix Media
-	Copyright (C) 2012 - 2014, Michael Larabel
+	Copyright (C) 2012 - 2018, Phoronix Media
+	Copyright (C) 2012 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -23,7 +23,7 @@
 class gui implements pts_option_interface
 {
 	const doc_section = 'Web / GUI Support';
-	const doc_description = 'Launch the Phoronix Test Suite HTML5 web user-interface in the local GUI mode (no remote web support) and attempt to auto-launch the web-browser.';
+	const doc_description = 'Launch the Phoronix Test Suite HTML5 web user-interface in the local GUI mode (no remote web support) and attempt to auto-launch the web-browser. THIS FEATURE IS CURRENTLY EXPERIMENTAL AND NO LONGER ACTIVE DEVELOPMENT. See Phoronix Test Suite Phoromatic as an alternative web UI approach.';
 
 	public static function command_aliases()
 	{
@@ -39,6 +39,10 @@ class gui implements pts_option_interface
 			return false;
 		}
 
+		echo pts_client::cli_just_bold(PHP_EOL . 'THE PHORONIX TEST SUITE WEB GUI IS CURRENTLY DEPRECATED AND UNMAINTAINED. NO FUTURE IMPROVEMENTS TO THIS GUI ARE PLANNED AT THIS TIME UNLESS THERE IS ENTERPRISE SUPPORT INTEREST.' . PHP_EOL . PHP_EOL . 'THOSE WANTING TO MAKE USE OF A PHORONIX TEST SUITE USER-INTERFACE ARE ENCOURAGED TO USE THE PHOROMATIC [https://www.phoromatic.com/] COMPONENT OF THE PHORONIX TEST SUITE. THERE IS ALSO `phoronix-test-suite interactive` and `phoronix-test-suite-shell` FOR A SELF-GUIDED PHORONIX TEST SUITE EXPERIENCE.' . PHP_EOL . PHP_EOL);
+		echo 'Continuing in 10 seconds...' . PHP_EOL;
+		sleep(10);
+
 		$web_port = 0;
 		$blocked_ports = array(2049, 3659, 4045, 6000);
 
@@ -104,14 +108,15 @@ class gui implements pts_option_interface
 			// Shutdown / Kill Servers (Might want to find a cleaner way)
 			$server_launcher .= PHP_EOL . 'taskkill /f /im php.exe';
 			// For now lets clean this up.
-			$server_launcher .= PHP_EOL . 'del /f ' . PTS_USER_PATH . 'run*' . PHP_EOL;
+			$server_launcher .= PHP_EOL . 'del /f ' . getenv('PTS_EXT_LAUNCH_SCRIPT_DIR') . 'run*' . PHP_EOL;
 			$server_launcher .= 'exit /B' . PHP_EOL . PHP_EOL;
 
 			// HTTP Server Setup
 			$server_launcher .= ':run_local_server' . PHP_EOL;
 			if(strpos(getenv('PHP_BIN'), 'hhvm'))
 			{
-				$server_launcher .= 'cd ' . PTS_CORE_PATH . 'web-interface\\ && ' . getenv('PHP_BIN') . ' --config ' . PTS_CORE_PATH . 'static\\hhvm-server.hdf -m server -vServer.Port=' . $web_port . ' -vServer.IP=' . $server_ip . ' -vServer.SourceRoot=' . PTS_CORE_PATH . 'web-interface\\' . ' &' . PHP_EOL;
+				echo PHP_EOL . 'Unfortunately, the HHVM built-in web server has abandoned upstream. Users will need to use the PHP binary or other alternatives.' . PHP_EOL . PHP_EOL;
+				return false;
 			}
 			else
 			{
@@ -131,20 +136,22 @@ class gui implements pts_option_interface
 		} 
 		else
 		{
-			$script_path = PTS_USER_PATH . 'web-server-launcher';
+			$script_path = getenv('PTS_EXT_LAUNCH_SCRIPT_DIR') . '/web-server-launcher';
 			pts_file_io::unlink($script_path);
 
 			$server_launcher = '#!/bin/sh' . PHP_EOL;
 
 			// WebSocket Server Setup
 			$server_launcher .= 'export PTS_WEBSOCKET_PORT=' . $web_socket_port . PHP_EOL;
+			$server_launcher .= 'export PTS_WEBSOCKET_SERVER=GUI' . PHP_EOL;
 			$server_launcher .= 'cd ' . getenv('PTS_DIR') . ' && PTS_MODE="CLIENT" ' . getenv('PHP_BIN') . ' pts-core/phoronix-test-suite.php start-ws-server &' . PHP_EOL;
 			$server_launcher .= 'websocket_server_pid=$!'. PHP_EOL;
 
 			// HTTP Server Setup
 			if(strpos(getenv('PHP_BIN'), 'hhvm'))
 			{
-				$server_launcher .= 'cd ' . PTS_CORE_PATH . 'web-interface/ && ' . getenv('PHP_BIN') . ' --config ' . PTS_CORE_PATH . 'static/hhvm-server.hdf -m server -vServer.Port=' . $web_port . ' -vServer.IP=' . $server_ip . ' -vServer.SourceRoot=' . PTS_CORE_PATH . 'web-interface/' . ' &' . PHP_EOL;
+				echo PHP_EOL . 'Unfortunately, the HHVM built-in web server has abandoned upstream. Users will need to use the PHP binary or other alternatives.' . PHP_EOL . PHP_EOL;
+				return false;
 			}
 			else
 			{

pts-core/commands/help.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2010 - 2011, Phoronix Media
-	Copyright (C) 2010 - 2011, Michael Larabel
+	Copyright (C) 2010 - 2017, Phoronix Media
+	Copyright (C) 2010 - 2017, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -23,11 +23,11 @@
 class help implements pts_option_interface
 {
 	const doc_section = 'Other';
-	const doc_description = 'This option will display the list of available Phoronix Test Suite commands.';
+	const doc_description = 'This option will display a list of available Phoronix Test Suite commands and possible parameter types.';
 
 	public static function run($r)
 	{
-		echo PHP_EOL . pts_title(true) . PHP_EOL . PHP_EOL;
+		echo PHP_EOL . pts_client::cli_colored_text(pts_core::program_title(true), 'green', true) . PHP_EOL . PHP_EOL;
 		echo pts_documentation::basic_description() . PHP_EOL . PHP_EOL . 'View the included PDF / HTML documentation or visit http://www.phoronix-test-suite.com/ for full details.' . PHP_EOL;
 		$options = pts_documentation::client_commands_array();
 
@@ -38,13 +38,15 @@ class help implements pts_option_interface
 				continue;
 			}
 
-			echo PHP_EOL . strtoupper($section) . PHP_EOL . PHP_EOL;
+			echo PHP_EOL . pts_client::cli_just_bold(strtoupper($section)) . PHP_EOL . PHP_EOL;
 
 			sort($contents);
+			$tabled = array();
 			foreach($contents as &$option)
 			{
-				echo '   ' . trim($option[0] . ' ' . implode(' ', $option[1])) . PHP_EOL;
+				$tabled[] = array($option[0], pts_client::cli_colored_text(implode(' ', $option[1]), 'gray'));
 			}
+			echo pts_user_io::display_text_table($tabled, '    ') . PHP_EOL;
 		}
 		echo PHP_EOL;
 	}

pts-core/commands/info.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2012, Phoronix Media
-	Copyright (C) 2008 - 2012, Michael Larabel
+	Copyright (C) 2008 - 2018, Phoronix Media
+	Copyright (C) 2008 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -28,123 +28,224 @@ class info implements pts_option_interface
 	public static function argument_checks()
 	{
 		return array(
-		new pts_argument_check(0, array('pts_types', 'identifier_to_object'), 'object')
+		new pts_argument_check(0, array('pts_types', 'identifier_to_object'))
 		);
 	}
 	public static function run($args)
 	{
 		echo PHP_EOL;
 
-		if($args['object'] instanceof pts_test_suite)
+		if($args[0] == 'pts/all')
 		{
-			pts_client::$display->generic_heading($args['object']->get_title());
-			echo 'Run Identifier: ' . $args['object']->get_identifier() . PHP_EOL;
-			echo 'Suite Version: ' . $args['object']->get_version() . PHP_EOL;
-			echo 'Maintainer: ' . $args['object']->get_maintainer() . PHP_EOL;
-			echo 'Suite Type: ' . $args['object']->get_suite_type() . PHP_EOL;
-			echo 'Unique Tests: ' . $args['object']->get_unique_test_count() . PHP_EOL;
-			echo 'Suite Description: ' . $args['object']->get_description() . PHP_EOL;
-			echo PHP_EOL;
-			echo $args['object']->pts_format_contained_tests_string();
-			echo PHP_EOL;
+			$args = pts_openbenchmarking::available_tests(false);
 		}
-		else if($args['object'] instanceof pts_test_profile)
+
+		foreach($args as $arg)
 		{
-			$test_title = $args['object']->get_title();
-			$test_version = $args['object']->get_app_version();
-			if(!empty($test_version))
+			$o = pts_types::identifier_to_object($arg);
+
+			if($o instanceof pts_test_suite)
 			{
-				$test_title .= ' ' . $test_version;
-			}
-
-			pts_client::$display->generic_heading($test_title);
-			echo 'Run Identifier: ' . $args['object']->get_identifier() . PHP_EOL;
-			echo 'Profile Version: ' . $args['object']->get_test_profile_version() . PHP_EOL;
-			echo 'Maintainer: ' . $args['object']->get_maintainer() . PHP_EOL;
-			echo 'Test Type: ' . $args['object']->get_test_hardware_type() . PHP_EOL;
-			echo 'Software Type: ' . $args['object']->get_test_software_type() . PHP_EOL;
-			echo 'License Type: ' . $args['object']->get_license() . PHP_EOL;
-			echo 'Test Status: ' . $args['object']->get_status() . PHP_EOL;
-			echo 'Project Web-Site: ' . $args['object']->get_project_url() . PHP_EOL;
-			if($args['object']->get_estimated_run_time() > 1)
-			{
-				echo 'Estimated Run-Time: ' . $args['object']->get_estimated_run_time() . ' Seconds' . PHP_EOL;
-			}
-
-			$download_size = $args['object']->get_download_size();
-			if(!empty($download_size))
-			{
-				echo 'Download Size: ' . $download_size . ' MB' . PHP_EOL;
-			}
-
-			$environment_size = $args['object']->get_environment_size();
-			if(!empty($environment_size))
-			{
-				echo 'Environment Size: ' . $environment_size . ' MB' . PHP_EOL;
-			}
-
-			echo PHP_EOL . 'Description: ' . $args['object']->get_description() . PHP_EOL;
-
-			if($args['object']->test_installation != false)
-			{
-				$last_run = $args['object']->test_installation->get_last_run_date();
-				$last_run = $last_run == '0000-00-00' ? 'Never' : $last_run;
-
-				$avg_time = $args['object']->test_installation->get_average_run_time();
-				$avg_time = !empty($avg_time) ? pts_strings::format_time($avg_time, 'SECONDS') : 'N/A';
-				$latest_time = $args['object']->test_installation->get_latest_run_time();
-				$latest_time = !empty($latest_time) ? pts_strings::format_time($latest_time, 'SECONDS') : 'N/A';
-
-				echo PHP_EOL . 'Test Installed: Yes' . PHP_EOL;
-				echo 'Last Run: ' . $last_run . PHP_EOL;
-
-				if($last_run != 'Never')
+				pts_client::$display->generic_heading($o->get_title());
+				echo pts_client::cli_just_bold('Suite Description: ') . ' ' . $o->get_description() . PHP_EOL . PHP_EOL;
+				$table = array();
+				$table[] = array(pts_client::cli_just_bold('Run Identifier:    '), $o->get_identifier());
+				$table[] = array(pts_client::cli_just_bold('Suite Version: '), $o->get_version());
+				$table[] = array(pts_client::cli_just_bold('Maintainer: '), $o->get_maintainer());
+				$table[] = array(pts_client::cli_just_bold('Suite Type: '), $o->get_suite_type());
+				$table[] = array(pts_client::cli_just_bold('Unique Tests: '), $o->get_unique_test_count());
+				$table[] = array(pts_client::cli_just_bold('Contained Tests: '));
+				foreach($o->get_contained_test_result_objects() as $result_obj)
 				{
-					if($args['object']->test_installation->get_run_count() > 1)
+					$table[] = array(pts_client::cli_just_bold(null), $result_obj->test_profile->get_title() . ' ', $result_obj->get_arguments_description());
+				}
+				echo pts_user_io::display_text_table($table) . PHP_EOL;
+			}
+			else if($o instanceof pts_test_profile)
+			{
+				$test_title = $o->get_title();
+				$test_version = $o->get_app_version();
+				if(!empty($test_version))
+				{
+					$test_title .= ' ' . $test_version;
+				}
+
+				pts_client::$display->generic_heading($test_title);
+
+				if($o->get_license() == 'Retail' || $o->get_license() == 'Restricted')
+				{
+					echo pts_client::cli_just_bold(strtoupper('NOTE: This test profile is marked \'' . $o->get_license() . '\' and may have issues running without third-party/commercial dependencies.')) . PHP_EOL . PHP_EOL;
+				}
+				if($o->get_status() != 'Verified')
+				{
+					echo pts_client::cli_just_bold(strtoupper('NOTE: This test profile is marked \'' . $o->get_status() . '\' and may have known issues with test installation or execution.')) . PHP_EOL . PHP_EOL;
+				}
+
+				$table = array();
+				$table[] = array(pts_client::cli_just_bold('Run Identifier: '), $o->get_identifier());
+				$table[] = array(pts_client::cli_just_bold('Profile Version: '), $o->get_test_profile_version());
+				$table[] = array(pts_client::cli_just_bold('Maintainer: '), $o->get_maintainer());
+				$table[] = array(pts_client::cli_just_bold('Test Type: '), $o->get_test_hardware_type());
+				$table[] = array(pts_client::cli_just_bold('Software Type: '), $o->get_test_software_type());
+				$table[] = array(pts_client::cli_just_bold('License Type: '), $o->get_license());
+				$table[] = array(pts_client::cli_just_bold('Test Status: '), $o->get_status());
+				$table[] = array(pts_client::cli_just_bold('Supported Platforms: '), implode(', ', $o->get_supported_platforms()));
+				$table[] = array(pts_client::cli_just_bold('Project Web-Site: '), $o->get_project_url());
+
+				if($o->get_estimated_run_time() > 1)
+				{
+					$table[] = array(pts_client::cli_just_bold('Estimated Run-Time: '), $o->get_estimated_run_time() . ' Seconds');
+				}
+
+				$download_size = $o->get_download_size();
+				if(!empty($download_size))
+				{
+					$table[] = array(pts_client::cli_just_bold('Download Size: '), $download_size . ' MB');
+				}
+
+				$environment_size = $o->get_environment_size();
+				if(!empty($environment_size))
+				{
+					$table[] = array(pts_client::cli_just_bold('Environment Size: '), $environment_size . ' MB');
+				}
+
+				echo pts_user_io::display_text_table($table);
+
+				echo PHP_EOL . PHP_EOL . pts_client::cli_just_bold('Description: ') . $o->get_description() . PHP_EOL;
+
+				if(stripos($o->get_identifier(0), 'local/') === false)
+				{
+					echo PHP_EOL . PHP_EOL . pts_client::cli_just_bold('OpenBenchmarking.org Test Profile: ') . 'https://openbenchmarking.org/test/' . $o->get_identifier() . PHP_EOL . PHP_EOL;
+				}
+
+				foreach(array('Pre-Install Message' => $o->get_pre_install_message(), 'Post-Install Message' => $o->get_post_install_message(), 'Pre-Run Message' => $o->get_pre_run_message(), 'Post-Run Message' => $o->get_post_run_message()) as $msg_type => $msg)
+				{
+					if($msg != null)
 					{
-						echo 'Average Run-Time: ' . $avg_time . PHP_EOL;
+						echo PHP_EOL . pts_client::cli_just_bold($msg_type . ': ') . $msg . PHP_EOL. PHP_EOL;
+					}
+				}
+
+				if($o->test_installation != false)
+				{
+					$last_run = $o->test_installation->get_last_run_date();
+					$last_run = $last_run == '0000-00-00' ? 'Never' : $last_run;
+
+					$avg_time = $o->test_installation->get_average_run_time();
+					$avg_time = !empty($avg_time) ? pts_strings::format_time($avg_time, 'SECONDS') : 'N/A';
+					$latest_time = $o->test_installation->get_latest_run_time();
+					$latest_time = !empty($latest_time) ? pts_strings::format_time($latest_time, 'SECONDS') : 'N/A';
+					$install_time = ceil($o->test_installation->get_latest_install_time());
+					$install_time = !empty($install_time) ? pts_strings::format_time($install_time, 'SECONDS') : 'N/A';
+
+					$table = array();
+					$table[] = array(pts_client::cli_just_bold('Test Installed: '), 'Yes');
+					$table[] = array(pts_client::cli_just_bold('Last Run: '), $last_run);
+					$table[] = array(pts_client::cli_just_bold('Install Time: '), $install_time);
+					if($o->test_installation->get_install_size() > 0)
+					{
+						$table[] = array(pts_client::cli_just_bold('Install Size: '), $o->test_installation->get_install_size() . ' Bytes');
 					}
 
-					echo 'Latest Run-Time: ' . $latest_time . PHP_EOL;
-					echo 'Times Run: ' . $args['object']->test_installation->get_run_count() . PHP_EOL;
+					if($last_run != 'Never')
+					{
+						if($o->test_installation->get_run_count() > 1)
+						{
+							$table[] = array(pts_client::cli_just_bold('Average Run-Time: '), $avg_time);
+						}
+
+						$table[] = array(pts_client::cli_just_bold('Latest Run-Time: '), $latest_time);
+						$table[] = array(pts_client::cli_just_bold('Times Run: '), $o->test_installation->get_run_count());
+					}
+					echo pts_user_io::display_text_table($table) . PHP_EOL;
+				}
+				else
+				{
+					echo PHP_EOL . pts_client::cli_just_bold('Test Installed: ') . 'No' . PHP_EOL;
+				}
+
+				$dependencies = $o->get_external_dependencies();
+				if(!empty($dependencies) && !empty($dependencies[0]))
+				{
+					echo PHP_EOL . pts_client::cli_just_bold('Software Dependencies:') . PHP_EOL;
+					echo pts_user_io::display_text_list($o->get_dependency_names());
+				}
+				echo PHP_EOL;
+
+
+
+				// OpenBenchmarking.org Change-Log
+				if(stripos($o->get_identifier(), 'local/') === false)
+				{
+
+					$change_log = pts_openbenchmarking_client::fetch_repository_test_profile_changelog($o->get_identifier(false));
+
+					if(is_array($change_log) && isset($change_log['tests'][$o->get_identifier_base_name()]['changes']))
+					{
+						echo pts_client::cli_just_bold('OpenBenchmarking.org Change History') . PHP_EOL;
+						$change_log = $change_log['tests'][$o->get_identifier_base_name()]['changes'];
+						foreach($change_log as $version => $data)
+						{
+							echo pts_client::cli_colored_text('v' . $version . ' - ' . date('j F Y', $data['last_updated']), 'green', true) . PHP_EOL;
+							echo $data['commit_description'] . PHP_EOL;
+						}
+					}
+				}
+
+				// Recent Test Results With This Test
+				$o_identifier = $o->get_identifier(false);
+				$table = array();
+				foreach(pts_client::saved_test_results() as $saved_results_identifier)
+				{
+					$result_file = new pts_result_file($saved_results_identifier);
+					foreach($result_file->get_result_objects() as $result_object)
+					{
+						if($result_object->test_profile->get_identifier(false) == $o_identifier)
+						{
+							$table[] = array(pts_client::cli_just_bold($result_file->get_identifier()), $result_file->get_title());
+							break;
+						}
+					}
+				}
+				if(!empty($table))
+				{
+					echo PHP_EOL . pts_client::cli_just_bold('Results Containing This Test') . PHP_EOL;
+					echo pts_user_io::display_text_table($table);
 				}
 			}
-			else
+			else if($o instanceof pts_result_file)
 			{
-				echo PHP_EOL . 'Test Installed: No' . PHP_EOL;
-			}
+				echo pts_client::cli_just_bold('Title: ') . $o->get_title() . PHP_EOL . pts_client::cli_just_bold('Identifier: ') . $o->get_identifier() . PHP_EOL;
+				echo PHP_EOL . pts_client::cli_just_bold('Test Result Identifiers:') . PHP_EOL;
+				echo pts_user_io::display_text_list($o->get_system_identifiers());
 
-			$dependencies = $args['object']->get_dependencies();
-			if(!empty($dependencies) && !empty($dependencies[0]))
-			{
-				echo PHP_EOL . 'Software Dependencies:' . PHP_EOL;
-				echo pts_user_io::display_text_list($args['object']->get_dependency_names());
-			}
-			echo PHP_EOL;
-		}
-		else if($args['object'] instanceof pts_result_file)
-		{
-			echo 'Title: ' . $args['object']->get_title() . PHP_EOL . 'Identifier: ' . $args['object']->get_identifier() . PHP_EOL;
-			echo PHP_EOL . 'Test Result Identifiers:' . PHP_EOL;
-			echo pts_user_io::display_text_list($args['object']->get_system_identifiers());
+				$test_titles = array();
+				foreach($o->get_result_objects() as $result_object)
+				{
+					if($result_object->test_profile->get_display_format() == 'BAR_GRAPH')
+					{
+						$test_titles[] = $result_object->test_profile->get_title();
+					}
+				}
 
-			if(count(($tests = $args['object']->get_unique_test_titles())) > 0)
-			{
-				echo PHP_EOL . 'Contained Tests:' . PHP_EOL;
-				echo pts_user_io::display_text_list($tests);
+				if(count($test_titles) > 0)
+				{
+					echo PHP_EOL . pts_client::cli_just_bold('Contained Tests:') . PHP_EOL;
+					echo pts_user_io::display_text_list(array_unique($test_titles));
+				}
+				echo PHP_EOL;
 			}
-			echo PHP_EOL;
-		}
-		else if($args['object'] instanceof pts_virtual_test_suite)
-		{
-			pts_client::$display->generic_heading($args['object']->get_title());
-			echo 'Virtual Suite Description: ' . $args['object']->get_description() . PHP_EOL . PHP_EOL;
+			else if($o instanceof pts_virtual_test_suite)
+			{
+				pts_client::$display->generic_heading($o->get_title());
+				echo pts_client::cli_just_bold('Virtual Suite Description: ') . $o->get_description() . PHP_EOL . PHP_EOL;
 
-			foreach($args['object']->get_contained_test_profiles() as $test_profile)
-			{
-				echo '- ' . $test_profile . PHP_EOL;
+				foreach($o->get_contained_test_profiles() as $test_profile)
+				{
+					echo '- ' . $test_profile . PHP_EOL;
+				}
+				echo PHP_EOL;
 			}
-			echo PHP_EOL;
 		}
 	}
 }

pts-core/commands/inspect_test_profile.php

@@ -0,0 +1,49 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2018, Phoronix Media
+	Copyright (C) 2018, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class inspect_test_profile implements pts_option_interface
+{
+	const doc_section = 'Asset Creation';
+	const doc_description = 'This option can be used for inspecting a Phoronix Test Suite test profile with providing inside details on test profiles for debugging / evaluation / learning purposes.';
+
+	public static function run($r)
+	{
+		$types = pts_validation::process_xsd_types();
+		foreach(pts_types::identifiers_to_test_profile_objects($r, true, true) as $test_profile)
+		{
+			pts_client::$display->generic_heading($test_profile . ' - test-definition.xml');
+			pts_validation::process_xsd_display_chart(pts_openbenchmarking::openbenchmarking_standards_path() . 'schemas/test-profile.xsd', $test_profile, $types);
+			pts_client::$display->generic_heading($test_profile . ' - downloads.xml');
+			pts_validation::process_xsd_display_chart(pts_openbenchmarking::openbenchmarking_standards_path() . 'schemas/test-profile-downloads.xsd', $test_profile, $types);
+			pts_client::$display->generic_heading($test_profile . ' - results-definition.xml');
+			pts_validation::process_xsd_display_chart(pts_openbenchmarking::openbenchmarking_standards_path() . 'schemas/results-parser.xsd', $test_profile, $types);
+		}
+	}
+	public static function argument_checks()
+	{
+		return array(
+		new pts_argument_check(0, array('pts_test_profile', 'is_test_profile'), null)
+		);
+	}
+}
+
+?>

pts-core/commands/install_dependencies.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2010, Phoronix Media
-	Copyright (C) 2008 - 2010, Michael Larabel
+	Copyright (C) 2008 - 2015, Phoronix Media
+	Copyright (C) 2008 - 2015, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -34,7 +34,6 @@ class install_dependencies implements pts_option_interface
 	public static function run($r)
 	{
 		$test_profiles = pts_types::identifiers_to_test_profile_objects($r, true, true);
-		//pts_client::set_test_flags(pts_c::auto_mode);
 		pts_external_dependencies::install_dependencies($test_profiles);
 	}
 }

pts-core/commands/install_test.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2011, Phoronix Media
-	Copyright (C) 2008 - 2011, Michael Larabel
+	Copyright (C) 2008 - 2018, Phoronix Media
+	Copyright (C) 2008 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -24,16 +24,12 @@ class install_test implements pts_option_interface
 {
 	const doc_section = 'Test Installation';
 	const doc_use_alias = 'install';
-	const doc_description = 'This option will install the selected test(s) inside the testing environment directory). The install process from downloading of the test files to the installation is fully automated. The install option needs to be supplied with the test name or suite as an argument. Optionally, a OpenBenchmarking.org ID or the name of a saved results file can be supplied as well and the test(s) to install will automatically be extracted from that information. If the test is already installed and was run by the latest version of the installation process, no action will be taken. Multiple arguments can be supplied to install additional tests at the same time.';
+	const doc_description = 'This option will install the selected test(s) inside the testing environment directory. The install process from downloading of the test files to the installation is fully automated. The install option needs to be supplied with the test name or suite as an argument. Optionally, a OpenBenchmarking.org ID or the name of a saved results file can be supplied as well and the test(s) to install will automatically be extracted from that information. If the test is already installed and was run by the latest version of the installation process, no action will be taken. Multiple arguments can be supplied to install additional tests at the same time.';
 
 	public static function command_aliases()
 	{
 		return array('install');
 	}
-	public static function invalid_command($passed_args = null)
-	{
-		pts_tests::invalid_command_helper($passed_args);
-	}
 	public static function argument_checks()
 	{
 		return array(

pts-core/commands/interactive.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2011 - 2013, Phoronix Media
-	Copyright (C) 2011 - 2013, Michael Larabel
+	Copyright (C) 2011 - 2018, Phoronix Media
+	Copyright (C) 2011 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -27,63 +27,27 @@ class interactive implements pts_option_interface
 
 	public static function run($r)
 	{
-		$is_moscow = pts_flags::os_identifier_hash() == 'b28d6a7148b34595c5b397dfcf5b12ac7932b3dc';
-
-		if($is_moscow)
-		{
-			// Auto mount?
-			$drives = pts_file_io::glob('/dev/sda*');
-			sort($drives);
-
-			if(false && count($drives) > 0 && !is_dir('/media/pts-auto-mount') && is_writable('/media/'))
-			{
-				$last_drive = array_pop($drives);
-				echo PHP_EOL . 'Attempting to auto-mount drive: ' . $last_drive . PHP_EOL;
-				mkdir('/media/pts-auto-mount');
-				exec('mount ' . $last_drive . ' /media/pts-auto-mount');
-				putenv('PTS_TEST_INSTALL_ROOT_PATH=/media/pts-auto-mount/');
-			}
-
-			// Auto save results
-			$test_results_name = phodevi::read_property('motherboard', 'serial-number');
-
-			if($test_results_name == null)
-			{
-				$test_results_name = phodevi::read_name('motherboard');
-			}
-			if($test_results_name == null)
-			{
-				$test_results_name = phodevi::read_property('system', 'vendor-identifier');
-			}
-
-			putenv('TEST_RESULTS_NAME=' . str_replace(' ', null, $test_results_name));
-			putenv('TEST_RESULTS_IDENTIFIER=' . $test_results_name);
-			putenv('TEST_RESULTS_DESCRIPTION=Tests using ' . phodevi::read_property('system', 'operating-system') . ' on ' . date('d F Y') . ' of ' . $test_results_name . '.');
-			self::select_drive_mount();
-		}
-
 		pts_openbenchmarking::refresh_repository_lists();
 		pts_client::$display->generic_heading('Interactive Benchmarking');
-		echo 'System Hardware:' . PHP_EOL . phodevi::system_hardware(true) . (phodevi::read_property('motherboard', 'serial-number') != null ? PHP_EOL . 'System Serial Number: ' . phodevi::read_property('motherboard', 'serial-number') : null) . PHP_EOL . PHP_EOL . PHP_EOL;
-		$reboot_on_exit = pts_flags::is_live_cd() && pts_client::user_home_directory() == '/root/';
+		echo phodevi::system_centralized_view();
+		echo PHP_EOL . (phodevi::read_property('motherboard', 'serial-number') != null ? PHP_EOL . 'System Serial Number: ' . phodevi::read_property('motherboard', 'serial-number') : null) . PHP_EOL . PHP_EOL;
+		$reboot_on_exit = false;
 
 		do
 		{
+			$only_show_available_cached_tests = pts_network::internet_support_available() == false;
 			$options = array(
-				'RUN_TEST' => 'Run A Test',
-				'RUN_SUITE' => 'Run A Suite [A Collection Of Tests]',
-				'RUN_SYSTEM_TEST' => 'Run Complex System Test',
+				'RUN_TEST' => 'Run A Test / Benchmark',
+				'RUN_SUITE' => 'Run A Suite      [A Collection Of Tests]',
+				'RUN_STRESS_TEST' => 'Run A Stress Test      [Concurrent Benchmarks For Heavy System Load]',
 				'SHOW_INFO' => 'Show System Hardware / Software Information',
-				'SHOW_SENSORS' => 'Show Auto-Detected System Sensors',
-				'SET_RUN_COUNT' => 'Set Test Run Repetition'
+				'SHOW_SENSORS' => 'Show Available System Sensors',
+				'LIST_TESTS' => 'List Available Tests',
+				'LIST_RECOMMENDED_TESTS' => 'List Recommended Tests',
+			//	'SET_RUN_COUNT' => 'Set Test Run Repetition',
+				'SEARCH' => 'Search Tests / Suites / Results'
 				);
 
-			if($is_moscow)
-			{
-				unset($options['RUN_SUITE']);
-			//	$options['SELECT_DRIVE_MOUNT'] = 'Select Disk Drive To Use For Testing';
-			}
-
 			if(count(pts_client::saved_test_results()) > 0)
 			{
 				$options['BACKUP_RESULTS_TO_USB'] = 'Backup Results To Media Storage';
@@ -95,20 +59,19 @@ class interactive implements pts_option_interface
 			switch($response)
 			{
 				case 'RUN_TEST':
-					$supported_tests = pts_openbenchmarking::available_tests();
+					$supported_tests = pts_openbenchmarking::available_tests(!$only_show_available_cached_tests, false, false, false, $only_show_available_cached_tests);
 					$supported_tests = pts_types::identifiers_to_test_profile_objects($supported_tests, false, true);
 					$longest_title_length = 0;
 
 					foreach($supported_tests as $i => &$test_profile)
 					{
-						if($test_profile->get_title() == null || pts_test_run_manager::test_profile_system_compatibility_check($test_profile) == false)
+						if($test_profile->get_title() == null || $test_profile->get_license() == 'Retail')
 						{
 							unset($supported_tests[$i]);
 							continue;
 						}
-						if($is_moscow && pts_test_install_request::test_files_available_locally($test_profile) == false)
+						if(!pts_test_run_manager::test_profile_system_compatibility_check($test_profile))
 						{
-							// Don't show tests where files need to be downloaded
 							unset($supported_tests[$i]);
 							continue;
 						}
@@ -127,54 +90,92 @@ class interactive implements pts_option_interface
 					$supported_tests = $t;
 					asort($supported_tests);
 
-					$tests_to_run = pts_user_io::prompt_text_menu('Select Test', $supported_tests, true, true);
+					$tests_to_run = pts_user_io::prompt_text_menu('Select Test(s)', $supported_tests, true, true);
 					$tests_to_run = explode(',', $tests_to_run);
 					pts_test_installer::standard_install($tests_to_run);
-					$run_manager = pts_test_run_manager::standard_run($tests_to_run, (pts_c::defaults_mode | pts_c::auto_mode));
+					$run_manager = new pts_test_run_manager(false, 2);
+					$run_manager->standard_run($tests_to_run);
 					if($run_manager != false)
 					{
 						pts_client::display_web_page(PTS_SAVE_RESULTS_PATH . $run_manager->get_file_name() . '/index.html', null, true, true);
 					}
 					break;
+				case 'RUN_STRESS_TEST':
+					$supported_tests = pts_openbenchmarking::available_tests(!$only_show_available_cached_tests, false, false, false, $only_show_available_cached_tests);
+					$supported_tests = pts_types::identifiers_to_test_profile_objects($supported_tests, false, true);
+					$longest_title_length = 0;
+
+					foreach($supported_tests as $i => &$test_profile)
+					{
+						if($test_profile->get_title() == null || $test_profile->get_license() == 'Retail')
+						{
+							unset($supported_tests[$i]);
+							continue;
+						}
+						if(!pts_test_run_manager::test_profile_system_compatibility_check($test_profile))
+						{
+							unset($supported_tests[$i]);
+							continue;
+						}
+
+						$longest_title_length = max($longest_title_length, strlen($test_profile->get_title()));
+					}
+
+					$t = array();
+					foreach($supported_tests as $i => &$test_profile)
+					{
+						if($test_profile instanceof pts_test_profile)
+						{
+							$t[$test_profile->get_identifier()] = sprintf('%-' . ($longest_title_length + 1) . 'ls - %-10ls', $test_profile->get_title(), $test_profile->get_test_hardware_type());
+						}
+					}
+					$supported_tests = $t;
+					asort($supported_tests);
+
+					$tests_to_run = pts_user_io::prompt_text_menu('Select Test(s)', $supported_tests, true, true);
+					$tests_to_run = explode(',', $tests_to_run);
+
+					$concurrent_runs = pts_user_io::prompt_user_input('Number of tests to run concurrently');
+					putenv('PTS_CONCURRENT_TEST_RUNS=' . trim($concurrent_runs));
+					$minutes_loop_time = pts_user_io::prompt_user_input('Number of minutes to stress run');
+					putenv('TOTAL_LOOP_TIME=' . trim($minutes_loop_time));
+
+					pts_test_installer::standard_install($tests_to_run);
+					pts_client::execute_command('stress_run', $tests_to_run);
+					break;
 				case 'RUN_SUITE':
 					$possible_suites = pts_openbenchmarking::available_suites();
 
 					foreach(array_map('strtolower', pts_types::subsystem_targets()) as $subsystem)
 					{
-						array_push($possible_suites, 'pts/' . $subsystem);
+						$possible_suites[] = 'pts/' . $subsystem;
 					}
 
 					$suites_to_run = pts_user_io::prompt_text_menu('Select Suite', $possible_suites, true);
 					foreach(explode(',', $suites_to_run) as $suite_to_run)
 					{
 						pts_test_installer::standard_install($suite_to_run);
-						pts_test_run_manager::standard_run($suite_to_run, (pts_c::defaults_mode | pts_c::auto_mode));
+						$run_manager = new pts_test_run_manager(false, 2);
+						$run_manager->standard_run($suite_to_run);
 					}
 					break;
 				case 'SELECT_DRIVE_MOUNT':
 					self::select_drive_mount();
 					break;
-				case 'RUN_SYSTEM_TEST':
-					pts_client::$display->generic_heading('System Test');
-					$system_tests = array('apache', 'c-ray', 'ramspeed', 'postmark');
-					pts_test_installer::standard_install($system_tests);
-					$run_manager = pts_test_run_manager::standard_run($system_tests, (pts_c::defaults_mode | pts_c::auto_mode));
-					if($run_manager != false)
-					{
-						pts_client::display_web_page(PTS_SAVE_RESULTS_PATH . $run_manager->get_file_name() . '/index.html', null, true, true);
-					}
+				case 'SEARCH':
+					pts_client::execute_command('search');
 					break;
 				case 'SHOW_INFO':
-					pts_client::$display->generic_heading('System Software / Hardware Information');
-					echo 'Hardware:' . PHP_EOL . phodevi::system_hardware(true) . PHP_EOL . PHP_EOL;
-					echo 'Software:' . PHP_EOL . phodevi::system_software(true) . PHP_EOL . PHP_EOL;
+					pts_client::execute_command('system_info');
 					break;
 				case 'SHOW_SENSORS':
-					pts_client::$display->generic_heading('Detected System Sensors');
-					foreach(phodevi::supported_sensors() as $sensor)
-					{
-						echo phodevi::sensor_name($sensor) . ': ' . phodevi::read_sensor($sensor) . ' ' . phodevi::read_sensor_unit($sensor) . PHP_EOL;
-					}
+					pts_client::execute_command('system_sensors');
+					break;
+				case 'LIST_TESTS':
+					pts_client::execute_command('list_available_tests');
+					break;
+				case 'LIST_RECOMMENDED_TESTS':
+					pts_client::execute_command('list_recommended_tests');
 					break;
 				case 'SET_RUN_COUNT':
 					$run_count = pts_user_io::prompt_user_input('Set the minimum number of times each test should repeat', false);
@@ -182,39 +183,20 @@ class interactive implements pts_option_interface
 					break;
 				case 'BACKUP_RESULTS_TO_USB':
 					pts_client::$display->generic_heading('Backing Up Test Results');
-
-					if($is_moscow)
+					$writable_backup_locations = array();
+					foreach(array_merge(pts_file_io::glob('/media/*'), pts_file_io::glob('/run/media/*/*')) as $media_dir)
 					{
-						$drives = pts_file_io::glob('/dev/sd*');
-						sort($drives);
-
-						if(count($drives) > 0 && is_writable('/media/'))
+						if(is_writable($media_dir))
 						{
-							$select_drive = pts_user_io::prompt_text_menu('Select Drive / Partition To Save Results', $drives);
-							echo PHP_EOL . 'Attempting to mount: ' . $select_drive . PHP_EOL;
-							mkdir('/media/00-results-backup');
-							exec('mount ' . $select_drive . ' /media/00-results-backup');
+							$writable_backup_locations[] = $media_dir;
 						}
 					}
 
-					foreach(pts_file_io::glob('/media/*') as $media_dir)
-					{
-						if(!is_writable($media_dir))
-						{
-							echo PHP_EOL . $media_dir . ' is not writable.' . PHP_EOL;
-							continue;
-						}
-
-						echo PHP_EOL . 'Writing Test Results To: ' . $media_dir . PHP_EOL;
-						pts_file_io::copy(PTS_SAVE_RESULTS_PATH, $media_dir . '/');
-						break;
-					}
-
-					if($is_moscow && is_dir('/media/00-results-backup'))
-					{
-						exec('umount /media/00-results-backup');
-						rmdir('/media/00-results-backup');
-					}
+					$backup_location = pts_user_io::prompt_text_menu('Select Backup Location', $writable_backup_locations);
+					$backup_location .= '/phoronix-test-suite-test-results/';
+					pts_file_io::mkdir($backup_location);
+					echo PHP_EOL . pts_client::cli_just_bold('Writing Test Results To: ') . $backup_location . PHP_EOL;
+					pts_file_io::copy(PTS_SAVE_RESULTS_PATH, $backup_location . '/');
 					break;
 			}
 			echo PHP_EOL . PHP_EOL;
@@ -229,7 +211,7 @@ class interactive implements pts_option_interface
 				exec('umount /media/pts-auto-mount 2>&1');
 			}
 
-			exec('reboot');
+			phodevi::reboot();
 		}
 	}
 	private static function select_drive_mount()
@@ -242,7 +224,7 @@ class interactive implements pts_option_interface
 		}
 		else
 		{
-			array_push($drives, 'No HDD');
+			$drives[] = 'No HDD';
 			$to_mount = pts_user_io::prompt_text_menu('Select Drive / Partition To Mount', $drives);
 
 			if($to_mount != 'No HDD')

pts-core/commands/internal_run.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2013, Phoronix Media
-	Copyright (C) 2013, Michael Larabel
+	Copyright (C) 2013 - 2018, Phoronix Media
+	Copyright (C) 2013 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -33,7 +33,7 @@ class internal_run implements pts_option_interface
 	}
 	public static function run($r)
 	{
-		pts_test_run_manager::set_batch_mode(array(
+		$test_run_manager = new pts_test_run_manager(array(
 			'UploadResults' => false,
 			'SaveResults' => true,
 			'PromptForTestDescription' => true,
@@ -43,11 +43,7 @@ class internal_run implements pts_option_interface
 			'OpenBrowser' => true
 			));
 
-		pts_test_run_manager::standard_run($r, pts_c::batch_mode);
-	}
-	public static function invalid_command($passed_args = null)
-	{
-		pts_tests::invalid_command_helper($passed_args);
+		$test_run_manager->standard_run($r);
 	}
 }
 

pts-core/commands/list_all_tests.php

@@ -0,0 +1,80 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2008 - 2018, Phoronix Media
+	Copyright (C) 2008 - 2018, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class list_all_tests implements pts_option_interface
+{
+	const doc_section = 'Information';
+	const doc_description = 'This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories. Unlike the other test listing options, list-all-tests will show deprecated tests, potentially broken tests, or other tests not recommended for all environments. The only check in place is ensuring the test profiles are at least compatible with the operating system in use.';
+
+	public static function run($r)
+	{
+		pts_client::$display->generic_heading('Available Tests');
+		$list_all_tests = pts_client::get_sent_command() == 'list_all_tests';
+		$only_show_available_cached_tests = !$list_all_tests && pts_network::internet_support_available() == false;
+
+		if($only_show_available_cached_tests)
+		{
+			echo 'Internet support is not available/enabled, so the Phoronix Test Suite is only listing test profiles where any necessary test assets are already downloaded to the system or available via a network download cache. To override this behavior, use the ' . pts_client::cli_just_bold('phoronix-test-suite list-all-tests') . ' option.' . PHP_EOL . PHP_EOL;
+			pts_client::execute_command('list_cached_tests');
+			return true;
+		}
+
+		$test_count = 0;
+		foreach(pts_openbenchmarking::available_tests(false) as $identifier)
+		{
+			$repo = substr($identifier, 0, strpos($identifier, '/'));
+			$id = substr($identifier, strlen($repo) + 1);
+			$repo_index = pts_openbenchmarking::read_repository_index($repo);
+			if((!empty($repo_index['tests'][$id]['supported_platforms']) && !in_array(phodevi::os_under_test(), $repo_index['tests'][$id]['supported_platforms'])) || empty($repo_index['tests'][$id]['title']))
+			{
+				// Don't show unsupported tests
+				continue;
+			}
+			if($list_all_tests == false && !empty($repo_index['tests'][$id]['status']) && $repo_index['tests'][$id]['status'] != 'Verified')
+			{
+				// Don't show unsupported tests
+				continue;
+			}
+
+			echo sprintf('%-30ls - %-39ls %-9ls', $identifier, $repo_index['tests'][$id]['title'], $repo_index['tests'][$id]['test_type']) . PHP_EOL;
+			$test_count++;
+		}
+
+		foreach(pts_tests::local_tests() as $identifier)
+		{
+			$test_profile = new pts_test_profile($identifier);
+
+			if($test_profile->get_title() != null && $test_profile->is_supported(false))
+			{
+				echo sprintf('%-30ls - %-39ls %-9ls', $test_profile->get_identifier(), $test_profile->get_title(), $test_profile->get_test_hardware_type()) . PHP_EOL;
+				$test_count++;
+			}
+		}
+
+		if($test_count == 0)
+		{
+			echo PHP_EOL . 'No tests found. Please check that you have Internet connectivity to download test profile data from OpenBenchmarking.org. The Phoronix Test Suite has documentation on configuring the network setup, proxy settings, and PHP network options. Please contact Phoronix Media if you continuing to experience problems.' . PHP_EOL . PHP_EOL;
+		}
+	}
+}
+
+?>

pts-core/commands/list_available_suites.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2013, Phoronix Media
-	Copyright (C) 2008 - 2013, Michael Larabel
+	Copyright (C) 2008 - 2018, Phoronix Media
+	Copyright (C) 2008 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -31,7 +31,7 @@ class list_available_suites implements pts_option_interface
 	}
 	public static function run($r)
 	{
-		$available_suites = pts_openbenchmarking::available_suites(false);
+		$available_suites = pts_tests::all_suites();
 		pts_client::$display->generic_heading('Available Suites');
 
 		if(count($available_suites) > 0)

pts-core/commands/list_available_tests.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2013, Phoronix Media
-	Copyright (C) 2008 - 2013, Michael Larabel
+	Copyright (C) 2008 - 2018, Phoronix Media
+	Copyright (C) 2008 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -23,7 +23,7 @@
 class list_available_tests implements pts_option_interface
 {
 	const doc_section = 'Information';
-	const doc_description = 'This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories.';
+	const doc_description = 'This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state. If the system has no Internet access, it will only list the test profiles where the necesary test assets are available locally on the system or on an available network cache (the same behavior as using the list-cached-tests sub-command), unless using the list-all-tests option to override this behavior.';
 
 	public static function command_aliases()
 	{
@@ -32,30 +32,48 @@ class list_available_tests implements pts_option_interface
 	public static function run($r)
 	{
 		pts_client::$display->generic_heading('Available Tests');
+		$only_show_available_cached_tests = pts_network::internet_support_available() == false;
+
+		if($only_show_available_cached_tests)
+		{
+			echo 'Internet support is not available/enabled, so the Phoronix Test Suite is only listing test profiles where any necessary test assets are already downloaded to the system or available via a network download cache. To override this behavior, use the ' . pts_client::cli_just_bold('phoronix-test-suite list-all-tests') . ' option.' . PHP_EOL . PHP_EOL;
+			pts_client::execute_command('list_cached_tests');
+			return true;
+		}
+
 		$test_count = 0;
 		foreach(pts_openbenchmarking::available_tests(false) as $identifier)
 		{
 			$repo = substr($identifier, 0, strpos($identifier, '/'));
 			$id = substr($identifier, strlen($repo) + 1);
 			$repo_index = pts_openbenchmarking::read_repository_index($repo);
-
-			if(!in_array(phodevi::operating_system(), $repo_index['tests'][$id]['supported_platforms']) || empty($repo_index['tests'][$id]['title']))
+			if((!empty($repo_index['tests'][$id]['supported_platforms']) && !in_array(phodevi::os_under_test(), $repo_index['tests'][$id]['supported_platforms'])) || empty($repo_index['tests'][$id]['title']))
 			{
 				// Don't show unsupported tests
 				continue;
 			}
+			if(!empty($repo_index['tests'][$id]['status']) && $repo_index['tests'][$id]['status'] != 'Verified')
+			{
+				// Don't show unsupported tests
+				continue;
+			}
+			if($repo_index['tests'][$id]['test_type'] == 'Graphics' && !phodevi::is_display_server_active())
+			{
+				// Don't display graphics tests that can't run
+				continue;
+			}
 
-			echo sprintf('%-30ls - %-35ls %-9ls', $identifier, $repo_index['tests'][$id]['title'], $repo_index['tests'][$id]['test_type']) . PHP_EOL;
+			echo sprintf('%-30ls - %-39ls %-9ls', $identifier, $repo_index['tests'][$id]['title'], $repo_index['tests'][$id]['test_type']) . PHP_EOL;
 			$test_count++;
 		}
 
-		foreach(pts_file_io::glob(PTS_TEST_PROFILE_PATH . 'local/*/test-definition.xml') as $path)
+		foreach(pts_tests::local_tests() as $identifier)
 		{
-			$test_profile = new pts_test_profile('local/' . basename(dirname($path)));
+			$test_profile = new pts_test_profile($identifier);
 
 			if($test_profile->get_title() != null && $test_profile->is_supported(false))
 			{
-				echo sprintf('%-30ls - %-35ls %-9ls', $test_profile->get_identifier(), $test_profile->get_title(), $test_profile->get_test_hardware_type()) . PHP_EOL;
+				echo sprintf('%-30ls - %-39ls %-9ls', $test_profile->get_identifier(), $test_profile->get_title(), $test_profile->get_test_hardware_type()) . PHP_EOL;
 				$test_count++;
 			}
 		}

pts-core/commands/list_cached_tests.php

@@ -0,0 +1,99 @@
+<?php
+
+/*
+	Phoronix Test Suite
+	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+	Copyright (C) 2008 - 2018, Phoronix Media
+	Copyright (C) 2008 - 2018, Michael Larabel
+
+	This program is free software; you can redistribute it and/or modify
+	it under the terms of the GNU General Public License as published by
+	the Free Software Foundation; either version 3 of the License, or
+	(at your option) any later version.
+
+	This program is distributed in the hope that it will be useful,
+	but WITHOUT ANY WARRANTY; without even the implied warranty of
+	MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+	GNU General Public License for more details.
+
+	You should have received a copy of the GNU General Public License
+	along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+class list_cached_tests implements pts_option_interface
+{
+	const doc_section = 'Information';
+	const doc_description = 'This option will list all test profiles where any needed test profiles are already cached or available from the local system under test. This is primarily useful if testing offline/behind-the-firewall and other use-cases where wanting to rely only upon local data.';
+
+	public static function run($r)
+	{
+		pts_client::$display->generic_heading('Cached Tests');
+
+		$test_count = 0;
+		foreach(pts_openbenchmarking::available_tests(false) as $identifier)
+		{
+			$repo = substr($identifier, 0, strpos($identifier, '/'));
+			$id = substr($identifier, strlen($repo) + 1);
+			$repo_index = pts_openbenchmarking::read_repository_index($repo);
+			if((!empty($repo_index['tests'][$id]['supported_platforms']) && !in_array(phodevi::os_under_test(), $repo_index['tests'][$id]['supported_platforms'])) || empty($repo_index['tests'][$id]['title']))
+			{
+				// Don't show unsupported tests
+				continue;
+			}
+			if($repo_index['tests'][$id]['test_type'] == 'Graphics' && !phodevi::is_display_server_active())
+			{
+				// Don't display graphics tests that can't run
+				continue;
+			}
+
+			$show = false;
+			foreach($repo_index['tests'][$id]['versions'] as $version)
+			{
+				if(!pts_openbenchmarking::is_test_profile_downloaded($identifier . '-' . $version))
+				{
+					// Without Internet, won't be able to download test, so don't show it
+					continue;
+				}
+				$test_profile = new pts_test_profile($identifier . '-' . $version);
+				if(pts_test_install_request::test_files_available_via_cache($test_profile) == false)
+				{
+					// only show tests where files are local or in an available cache
+					continue;
+				}
+				$show = true;
+				$identifier .= '-' . $version;
+				break;
+			}
+			if($show == false)
+			{
+				continue;
+			}
+
+			echo sprintf('%-30ls - %-39ls %-9ls', $identifier, $repo_index['tests'][$id]['title'], $repo_index['tests'][$id]['test_type']) . PHP_EOL;
+			$test_count++;
+		}
+
+		foreach(pts_tests::local_tests() as $identifier)
+		{
+			$test_profile = new pts_test_profile($identifier);
+			if(pts_test_install_request::test_files_available_via_cache($test_profile) == false)
+			{
+				// only show tests where files are local or in an available cache
+				continue;
+			}
+
+			if($test_profile->get_title() != null && $test_profile->is_supported(false))
+			{
+				echo sprintf('%-30ls - %-39ls %-9ls', $test_profile->get_identifier(), $test_profile->get_title(), $test_profile->get_test_hardware_type()) . PHP_EOL;
+				$test_count++;
+			}
+		}
+
+		if($test_count == 0)
+		{
+			echo PHP_EOL . 'No tests found. Please check that you have Internet connectivity to download test profile data from OpenBenchmarking.org. The Phoronix Test Suite has documentation on configuring the network setup, proxy settings, and PHP network options. Please contact Phoronix Media if you continuing to experience problems.' . PHP_EOL . PHP_EOL;
+		}
+	}
+}
+
+?>

pts-core/commands/list_installed_suites.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2009 - 2013, Phoronix Media
-	Copyright (C) 2009 - 2013, Michael Larabel
+	Copyright (C) 2009 - 2016, Phoronix Media
+	Copyright (C) 2009 - 2016, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by
@@ -34,7 +34,7 @@ class list_installed_suites implements pts_option_interface
 			$suite = new pts_test_suite($suite);
 			if($suite->needs_updated_install() == false)
 			{
-				array_push($installed_suites, $suite);
+				$installed_suites[] = $suite;
 			}
 		}