Prep for 9.0 M3

pts-core: Split OpenBenchmarking.org upload code into pts_openbenchmarking_upload file and allow it to be removed if wanting to strip out upload support

AUTHORS

@@ -1,4 +1,4 @@
-Copyright 2008 - 2018 by Phoronix Media.
+Copyright 2008 - 2019 by Phoronix Media.
 
 Lead Architects:
 

ChangeLog

@@ -1,5 +1,341 @@
 PHORONIX TEST SUITE CHANGE-LOG
 
+Phoronix Test Suite 9.0.0 Milestone 3
+3 September 2019
+
+pts-core: Add AllowResultUploadsToOpenBenchmarking user config option
+pts-core: Split OpenBenchmarking.org upload code into pts_openbenchmarking_upload file and allow it to be removed if wanting to strip out upload support
+pts-core: Add "ob-cache" local cache of tests/suites to help offline use-cases
+pts-core: Add hdf5 to the external dependency list
+pts-core: Add libxml2 to the external dependency list
+
+Phoronix Test Suite 9.0.0 Milestone 2
+22 August 2019
+
+pts-core: New result viewer improvements
+pts-core: Various fixes and code cleanups
+phodevi: Add support for cpu.power sensor reading from Linux zenpower driver
+
+Phoronix Test Suite 9.0.0 Milestone 1
+25 July 2019
+
+pts-core: Add 'expat' as possible external dependency
+pts-core: Various PDF report improvements
+pts-core: User-config syntax change LoadModules -> AutoLoadModules
+pts-core: Also dynamically increase run count if any results outside 3-sigma limit
+pts-core: Add compare-results-two-way sub-command
+pts-core: Improve MultiMatch in test result parser
+pts-core: Add IGNORE_RUNS environment variable
+pts-core: Add result-file-confidence sub-command
+pts_Graph: Add side-by-side 2-way comparison graph
+pts_Graph: Fix for displaying line graphs with very small values
+phodevi: Support reading sys.power via IPMI PS1_Input_Power/PS2_Input_Power
+phodevi: More robust Arm CPU model detection improvements
+phodevi: Much faster CPU usage reading on large core count systems
+
+Phoronix Test Suite 8.8.1-Hvaler
+30 May 2019
+
+pts-core: Various minor tweaks/fixes
+phodevi: Fix Linux CPU thread detection in some scenarios where inner-cores may be offlined
+phodevi: More robust CPU vulnerability reporting
+phodevi: Add initial reporting for Windows vulnerabilities via mdstool-cli
+
+Phoronix Test Suite 8.8.0-Hvaler
+13 May 2019
+
+pts-core: Report on the POWER SMT setting in notes
+pts-core: Add new/updated suite reporting
+pts-core: In build-suite allow not binding test profile versions
+pts-core: For list-suites, only show suites that have been updated within the past four years
+pts_Graph: Report N on horizontal bar graphs
+phodevi: Intel Node Manager power_meter support for sys.power sensor
+
+Phoronix Test Suite 8.8.0 Milestone 3
+9 May 2019
+
+pts-core: Add experimental dynamic-result-viewer
+pts-core: Allow marking of test profile download files as optional
+pts-core: Add "everything" virtual suite
+pts-core: Set MESA_VK_WSI_PRESENT_MODE=immediate by default to work around possible 60 FPS RADV bug
+pts-core: Rename remove-from-result-file to remove-run-from-result-file
+pts-core: Add remove-results-from-result-file sub-command
+phodevi: Detect AVX-512 VNNI
+phodevi: Fix OpenJDK Java version detection
+phodevi: Expand memory temperature sensor to work for some ARM SBCs
+phodevi: Multi-monitor detection improvements
+phodevi: Add support for gpu.memory-usage sensor with new AMDGPU mem_busy_percent sysfs interface
+system_monitor: Add PERFORMANCE_PER_SENSOR= support for performance-per-(arbitrary sensor) rather than just supporting perf-per-Watt
+ob_auto_compare: Show other results in result file on box plot
+
+Phoronix Test Suite 8.8.0 Milestone 2
+27 March 2019
+
+pts-core: On new installs set 3.00% as deviation for increasing dynamic run count rather than 3.50%
+pts-core: Add FORCE_MIN_TIMES_TO_RUN_CUTOFF= environment variable to toggle cutoff for FORCE_MIN_TIMES_TO_RUN behavior
+pts-core: Add FORCE_TIMES_TO_RUN_MULTIPLE
+pts-core: Add strict-benchmark option for enabling various options to ensure stricter behavior about the number of times to run each benchmark
+pts-core: Add cpu-physical-threads / cpu-threads / auto-executable auto test run options
+phodevi: CPU-therm reporting via sysfs for CPU core temp
+phodevi: GPU-therm reporting via sysfs for GPU core temp
+
+Phoronix Test Suite 8.8.0 Milestone 1
+16 March 2019
+
+pts-core: Allow FORCE_TIMES_TO_RUN to override debug-run/debug-benchmark single run behavior
+pts-core: Various fallbacks/workarounds for those wanting to (unofficially) use PTS on Windows 8.1
+pts-core: Other small tweaks and optimizations
+pts-core: Fixes an odd scenario where on Windows after 2 runs, further runs of that test profile for current run could fail in select (random) instances
+pts-core: Write out system notes for each system when generating PDF result output
+pts-core: CSV output improvements
+pts-core: Add result-file-raw-to-csv for dumping the raw/individual benchmark result run data
+pts_Graph: Condensing improvements for large graphs
+pts_Graph: Various formatting improvements
+phodevi: Intel GPU reporting improvements
+phodevi: BIOS version reporting support extended to Windows
+phodevi: ipmitool DMCI power sensor reporting for at least some POWER servers
+
+Phoronix Test Suite 8.6.1-Spydeberg
+21 February 2019
+
+pts-core: Dynamically honoring the CLI terminal width (no longer cache the value on Linux/BSD/macOS)
+pts-core: BSD external dependency updates
+ob_auto_compare: Minor formatting updates
+
+Phoronix Test Suite 8.6.0-Spydeberg
+5 February 2019
+
+pts-core: Other small tweaks and fixes
+pts-core: Fix for "/installed" virtual suite not including the proprietary/commercial tests that are installed
+ob_auto_compare: Allow showing more results in the embedded percentile graphs
+
+Phoronix Test Suite 8.6.0 Milestone 3
+27 January 2019
+
+pts-core: Better handling on Windows when there are space(s) in the paths
+pts-core: Add __event_post_run_stats module hook
+pts-core: Inline display of result distributions from OpenBenchmarking.org on applicable results
+pts-core: Don't try to run tests from result file(s) or suite(s) that were not installed
+pts-core: More accurate time reporting when result files are passed to `phoronix-test-suite estimate-run-time`
+phodevi: Better reporting of NVMe drive strings on BSDs
+
+Phoronix Test Suite 8.6.0 Milestone 2
+19 January 2019
+
+pts-core: Update Mac OS X to macOS references
+pts-core: Add result-file-stats sub-command to show more test result statistics
+pts-core: Show geometric mean and harmonic mean data at end of benchmarking process when relevant as part of post-run stats
+pts-core: Improved formatting of file download text user interface for downloading of dependencies on Microsoft Windows
+pts-core: Fix Loop 'Test i of n' Reporting
+pts-core: A few warning fixes for PHP 7.3+
+pts-core: External dependency handling support for OpenMandriva Lx 4.0 (transitioned from urpmi to DNF)
+phodevi: Report Windows Subsystem for Linux (WSL) to the system layer string
+
+Phoronix Test Suite 8.6.0 Milestone 1
+7 January 2019
+
+pts-core: No longer clear test profile identifiers on secondary saved results
+pts-core: Add compare-results-to-baseline sub-command for comparing one result file (baseline) against a set of results for text look at relative perf
+pts-core: Minor fixes
+pts-core: Show wins and losses summary after testing on large result files
+pts-core: On two-way result files, after test completion show comparison of current result against previous result (baseline)
+pts-core: Add ShowPostRunStatistics user configuration XML option to toggle whether various test statistics are printed at the end of testing
+pts-core: Add recently-added-tests sub-command to show newly added test profiles / alternative to openbenchmarking-changes/recently-updated-tests
+pts-core: Allow outputting line graphs to terminal interface as basic box plot graphs
+phodevi: Improve Linux multiple network device reporting
+phodevi: LXQt desktop version reporting
+phodevi: Update for Microsoft Windows Server 2019
+system_monitor: Allow outputting system monitor graphs to terminal interface inline
+
+Phoronix Test Suite 8.4.1-Skiptvet
+9 December 2018
+
+pts-core: Workaround for RHEL 7.6 PHP having buggy CURL that is causing seg faults
+pts-core: Ensure "." doesn't get set in PATH when launching phoronix-test-suite
+pts-core: Allow PRESET_OPTIONS env var to be used when in batch mode
+pts-core: FreeBSD updates
+pts-core: Add PRESET_OPTIONS_VALUES environment variable that is similar to PRESET_OPTIONS but for explicit values
+phodevi: Improved Radeon AMDGPU frequency detection/reporting
+phodevi: Add support for reading GPU usage sensor with AMDGPU gpu_busy_percent sysfs interface
+phodevi: Updated gpu_voltage sensor for AMDGPU driver
+phodevi: Updated gpu_power sensor for AMDGPU driver
+phodevi: Updated gpu_fanspeed sensor for AMDGPU driver
+phodevi: Updated gpu_freq sensor for AMDGPU driver
+phodevi: Improved physical CPU core detection on DragonFlyBSD 5.4
+phodevi: Improved Spectre detection on DragonFlyBSD 5.4
+phodevi: Allow using NUM_CPU_CORES to override the number of exposed CPU threads/cores to tests (alternative to PTS_NPROC / NUMBER_OF_PROCESSORS)
+phodevi: Improved multi-disk reporting for non-Linux operating systems
+phodevi: Minor GPU string reporting improvement
+
+Phoronix Test Suite 8.4.0-Skiptvet
+26 November 2018
+
+phodevi: Various minor tweaks
+
+Phoronix Test Suite 8.4.0 Milestone 3
+20 November 2018
+
+pts-core: Various external dependency updates
+pts-core: Expose NUM_CPU_PHYSICAL_CORES to test profiles as alternative to NUM_CPU_CORES (logical count)
+pts-core: Fix possible situation of openbenchmarking-refresh not updating when IndexCacheTTL is set to 0
+pts-core: Allow branded colors to work in the terminal/CLI text graphs
+pts-core: Continued POWER9 support improvements
+pts-core: Add RemoveInstallDirectoryOnReinstall to toggle test re-installation behavior
+pts_Graph: Fix for sometimes tick marker text overlapping
+phodevi: Improved physical CPU core detection for architectures like POWER
+phodevi: Improved CPU cache size detection
+
+Phoronix Test Suite 8.4.0 Milestone 2
+5 November 2018
+
+pts-core: Allow passing PTS environment variables as arguments to PTS and then just have PTS set them in the environment
+pts-core: Add dry-run test option
+pts-core: Various result parser additions to help parsing of CSV frame-time data for more test profiles (games)
+phodevi: Vulkan driver reporting fix
+
+Phoronix Test Suite 8.4.0 Milestone 1
+8 October 2018
+
+phodevi: Report vRAM size on GPU model string in GB when over 1GB
+pts-core: At run-time also report if test is deprecated/broken
+pts-core: Add "node-npm" as an external dependency for Node.js + NPM
+phodevi: Drop information reporting support for the long ago Catalyst/fglrx driver stack that hasn't been supported/maintained in many years
+
+Phoronix Test Suite 8.2.0-Rakkestad
+11 September 2018
+
+pts-core: Add RequiresDisplay test profile property to indicate if the test profile explicitly requires a monitor / display server to work
+pts-core: Add RequiresNetwork test profile property to indicate if the test profile needs a working network connection
+pts-core: Add RequiresInternet test profile property to indicate if the test profile needs a working Internet connection
+pts-core: Don't install tests where the current run-time checks will fail
+phodevi: Improve CPU thread count detection when running under LXC
+phodevi: More robust DMI decode ignore value handling
+phodevi: Add extra fallback detection for LXC in system-layer
+
+Phoronix Test Suite 8.2.0 Milestone 4
+7 September 2018
+
+pts-core: Fix behavior of "Would you like to stop and install these tests now"
+pts-core: Add reinstall / re-install aliases to force-install
+ob_auto_compare: Default first to trying to do an automated local result comparison
+phodevi: Amlogic SoC detection based on meson_cpufreq driver
+phodevi: Rockchip SoC fallback detection
+phodevi: Embedded ARM Cortex / ARM part model detection
+phodevi: Expanded sensor coverage using Intel RAPL
+
+Phoronix Test Suite 8.2.0 Milestone 3
+22 August 2018
+
+pts-core: Minor updates to logic for determining how many times to dynamically increase run count on high variance results
+pts-core: Show deprecated tests too when running list-all-tests
+pts-core: Add DropNoisyResults option if wishing to not save results that have too high of deviation between runs
+pts-core: Add PTS_DOWNLOAD_CACHE_OVERRIDE environment variable option on make-download-cache
+pts-core: Add PTS_USER_PATH_OVERRIDE environment variable to override PTS_USER_PATH on initialization
+pts-core: Add PTS_DOWNLOAD_CACHING_PLATFORM_LIMIT environment variable for make-download-cache
+pts-core: Add list-cached-tests option
+
+Phoronix Test Suite 8.2.0 Milestone 2
+15 August 2018
+
+pts-core: Add DivideResultDivisor to result parser
+pts-core: Add libevent as possible external dependency
+pts-core: Alert the user when the test install directory is on a file-system mounted with 'noexec'
+pts-core: Optional support with PHP 7.3+ to also obtain network info using net_get_interfaces()
+pts-core: Fix possible alignment issue with value/spacer on outputting text-based result graphs
+pts-core: When Internet is not available and running list-tests, only show tests that are available via caches unless `list-all-tests` is run
+pts-core: Rework `list-recommended-tests` to make use of download count information exposed via OB index repository data rather than needing to make fresh web query
+pts-core: Show "popular tests" in the invalid command helper
+pts-core: Allow stress-run to be triggered from interactive mode
+pts-core: Allow comparing dynamic in-line result comparisons against local results when no Internet results matching or Internet not available
+phodevi: Reporting of L1TF / Level 1 Terminal Fault vulnerability
+pts_Graph: Tweaks to text sizing on vertical bar graphs
+
+Phoronix Test Suite 8.2.0 Milestone 1
+21 July 2018
+
+pts-core: Add readline-based autocompletion support for text-based menu selection lists
+pts-core: Add readline-based autocompletion support for the result file save name for trying to match existing result file identifiers
+pts-core: Add autocompletion support for result file identifier based on installed software/hardware
+pts-core: Various Alpine Linux external dependency updates
+pts-core: Various improvements to the Shell sub-command functionality, including auto-completion support
+phoromatic: From rootadmin allow the rootadmin to override/reset a user account password
+phodevi: Windows battery reporting discharge rate support
+phodevi: Improved multi CPU socket reporting on Windows
+phodevi: Various hardware/software detection improvements when running within Docker/containers
+pgo: New module to provide for automated compiler PGO (Profile Guided Optimizations) testing on GCC/Clang
+
+Phoronix Test Suite 8.0.1-Aremark
+26 June 2018
+
+pts-core: Add "wine" external dependency support
+pts-core: Fix skip-test behavior
+pts-core: Web browser handling improvements on Windows, other Windows updates
+phoromatic: Various minor fixes and enhancements
+phodevi: Add USE_WINE environment variable check, point it towards the wine binary you wish to test
+
+Phoronix Test Suite 8.0.0-Aremark
+5 June 2018
+
+pts-core: Various macOS support tweaks/improvements
+pts-core: Add external dependency support for Brew macOS package manager
+pts-core: Avoid duplicate test identifiers on graphs
+phodevi: Improved X.Org Server version detection
+
+Phoronix Test Suite 8.0.0 Milestone 5
+30 May 2018
+
+pts-core: Termux external dependency support
+pts-core: Other minor tweaks and changes
+phoromatic: Better handling when the Phoromatic Server is run without an Internet connection or Internet disabled
+phoromatic: A wide variety of other Phoromatic Server optimizations and improvements
+phodevi: Add spec_store_bypass / Spectre Variant 4 reporting
+
+Phoronix Test Suite 8.0.0 Milestone 4
+16 May 2018
+
+phoromatic: Add a "email all users" option to the rootadmin page
+phoromatic: Improve search functionality on results page, allow boolean expressions, date ranges, many other search features
+phoromatic: Various search improvements to `phoronix-test-suite search`
+phoromatic: Allow new test profiles to be created from Phoromatic web UI
+pts-core: New create-test-profile sub-command for guided process to test profile creation
+pts-core: Basic changes, should be all that's needed for EU GDPR safeguards
+phodevi: RISC-V SiFive CPU detection
+
+Phoronix Test Suite 8.0.0 Milestone 3
+16 April 2018
+
+pts-core: Improvements for BSD / FreeBSD support
+pts-core: Add "V8" external dependency for V8 JavaScript engine
+pts-core: Add initial AdvertiseServiceOpenBenchmarkRelay support
+pts-core: Add new inspect-test-profile sub-command
+pts-core: Add ColoredConsole option in user configuration file of AUTO / TRUE / FALSE to control colored text behavior
+pts-core: Report each test's run-time to test-rin-times in result file XML
+pts-core: More XSD / OpenBenchmarking.org schema handling work for future functionality
+phoromatic: Ensure environment variables set via Phoromatic web interface get passed to tests
+phodevi: BSD improvements
+backup: A module providing initial backup/restore functionality for PTS/Phoromatic
+
+Phoronix Test Suite 8.0.0 Milestone 2
+24 March 2018
+
+pts-core: Add DeleteOutputBefore and DeleteOutputAfter options to the result parser
+pts-core: Fix for run-random-tests sub-command
+pts-core: Native XML result parsing support via FileFormat = XML in the result parser for test profiles
+pts-core: Add openbenchmarking-uploads sub-command for reporting recent result uploads from the local IP
+phodevi: reboot/shutdown support on Windows
+
+Phoronix Test Suite 8.0.0 Milestone 1
+8 March 2018
+
+pts-core: Rewritten Windows support
+pts-core: Add ~/install-message support where if a test profile installer writes there, that message will in turn be shown to the user on test install
+pts-core: Support for using Cygwin's bash interpreter on Windows systems for better test profile parsing support
+pts-core: When running on Windows, add new "windows/" OpenBenchmarking.org test repository
+pts-core: Introduce basic phoronix-test-suite shell for easier PTS experience
+pts-core: Allow XInclude support for test suites
+perf_per_dollar: Allow calculating cloud costs / perf-per via COST_PERF_PER_HOUR= environment variable
+phodevi: Cavium ThunderX detection
+
 Phoronix Test Suite 7.8.0-Folldal
 14 February 2018
 

README.md

@@ -1,13 +1,13 @@
-# Phoronix Test Suite 7.8.0m3
-http://www.phoronix-test-suite.com/
+# Phoronix Test Suite 9.0.0m3
+https://www.phoronix-test-suite.com/
 
 The **Phoronix Test Suite** is the most comprehensive testing and benchmarking
-platform available for Linux, Solaris, OS X, and BSD operating systems. The
-Phoronix Test Suite allows for carrying out tests in a fully automated manner
-from test installation to execution and reporting. All tests are meant to be
-easily reproducible, easy-to-use, and support fully automated execution. The
-Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed
-by Phoronix Media in cooperation with partners.
+platform available for Linux, Solaris, macOS, Windows, and BSD operating
+systems. The Phoronix Test Suite allows for carrying out tests in a fully
+automated manner from test installation to execution and reporting. All tests
+are meant to be easily reproducible, easy-to-use, and support fully automated
+execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license
+and is developed by Phoronix Media in cooperation with partners.
 
 The Phoronix Test Suite itself is an open-source framework for conducting
 automated tests along with reporting of test results, detection of installed
@@ -30,10 +30,10 @@ seamless collection of test profiles and/or additional test suites. A test
 profile consists of a set of Bash/shell scripts and XML files while a test suite
 is a single XML file.
 
-[OpenBenchmarking.org](http://www.openbenchmarking.org/) also allows for
+[OpenBenchmarking.org](https://www.openbenchmarking.org/) also allows for
 conducting side-by-side result comparisons, a central location for storing and
 sharing test results, and collaborating over test data.
-[Phoromatic](http://www.phoromatic.com/) is a complementary platform to
+[Phoromatic](https://www.phoromatic.com/) is a complementary platform to
 OpenBenchmarking.org and the Phoronix Test Suite for interfacing with Phoronix
 Test Suite client(s) to automatically execute test runs on a timed, per-commit,
 or other trigger-driven basis. Phoromatic is designed for enterprise and allows
@@ -42,7 +42,7 @@ Suite clients via a single web-based interface.
 
 Professional support and custom engineering for the Phoronix Test Suite,
 Phoromatic, and OpenBenchmarking.org is available by contacting
-<http://www.phoronix-test-suite.com/>.
+<https://www.phoronix-test-suite.com/>.
 
 Full details on the Phoronix Test Suite setup and usage is available from the
 included HTML/PDF documentation within the phoronix-test-suite package and from
@@ -50,11 +50,11 @@ the Phoronix Test Suite website.
 
 ## Installation & Setup
 
-The Phoronix Test Suite is supported on Linux, *BSD, Solaris, Mac OS X, and
-Windows systems. However, the most full-featured and well supported operating
-system for conducting the tests is Linux with some non-basic functionality not
-being available under all platforms. The Phoronix Test Suite software/framework
-is compatible with all major CPU architectures (e.g. i686, x86_64, ARM,
+The Phoronix Test Suite is supported on Linux, *BSD, Solaris, macOS, and Windows
+systems. However, the most full-featured and well supported operating system for
+conducting the tests is Linux with some non-basic functionality not being
+available under all platforms. The Phoronix Test Suite software/framework is
+compatible with all major CPU architectures (e.g. i686, x86_64, ARM, RISC-V,
 PowerPC), but not all of the test profiles/suites are compatible with all
 architectures.
 
@@ -63,7 +63,8 @@ without installation from the extracted tar.gz/zip package. The only hard
 dependency on the Phoronix Test Suite is having command-line support for PHP
 (PHP 5.3+) installed. A complete PHP stack (e.g. with web server) is **not**
 needed, but merely the PHP command-line support, which is widely available from
-operating system package managers under the name `php`, `php5-cli`, or `php5`.
+operating system package managers under the name `php`, `php7-cli`, `php5-cli`,
+or `php7`.
 
 ## Usage
 
@@ -81,11 +82,11 @@ Viewing installed system hardware and software is available via
 Facilitating a result comparison from OpenBenchmarking.org can be done by
 running, for example, `phoronix-test-suite benchmark 1204293-BY-PHORONIX357` if
 wishing to compare the results of the
-`http://openbenchmarking.org/result/1204293-BY-PHORONIX357` result file.
+`https://openbenchmarking.org/result/1204293-BY-PHORONIX357` result file.
 
 Additional information is available from the Phoronix Test Suite website
-<http://www.phoronix-test-suite.com/> and the material bundled within the
+<https://www.phoronix-test-suite.com/> and the material bundled within the
 `phoronix-test-suite/documentation/` directory or via
-<http://www.phoronix-test-suite.com/documentation/>. A man page is also bundled
+<https://www.phoronix-test-suite.com/documentation/>. A man page is also bundled
 with the phoronix-test-suite software.
 

deploy/deb-package/build-package-deb.php

@@ -3,8 +3,8 @@
 /*
 	Phoronix Test Suite
 	URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-	Copyright (C) 2008 - 2016, Phoronix Media
-	Copyright (C) 2008 - 2016, Michael Larabel
+	Copyright (C) 2008 - 2018, Phoronix Media
+	Copyright (C) 2008 - 2018, Michael Larabel
 
 	This program is free software; you can redistribute it and/or modify
 	it under the terms of the GNU General Public License as published by

documentation/TODO

@@ -13,55 +13,18 @@ PHORONIX TEST SUITE PUBLIC TODO LIST:
 - Optimus / DRI_PRIME=1 / xrandr --listproviders better GPU detection
 - NVIDIA make use of UsedDedicatedGPUMemory as new GPU sensor
 - Clean up all the WebSocket code and make it fit to spec
-- Incremental result uploads support
+- Incremental/streaming result uploads support
 - Check with Phodevi's parsers what other tools support JSON outputs - e.g. http://karelzak.blogspot.com/2015/06/json-output-for-basic-tools.html for cleaner parsing
-
-- Figure out why TestClientVersion in result file XML is missing after adding multiple systems, the previous ones are then blank
-- Restore the ReferenceID support in the results XML
 - Improve titles on tables like in: http://openbenchmarking.org/result/1603172-GA-AMDTONGA973
 
 ### PTS-GRAPH ###
 
-- Rotated text alignment for some PNG graphixs is sometimes off by 90/180 degrees compared to SVG, e.g. http://openbenchmarking.org/s/Compiler
+- Rotated text alignment for some PNG graphics is sometimes off by 90/180 degrees compared to SVG, e.g. http://openbenchmarking.org/s/Compiler
 
 ### OPENBENCHMARKING.ORG ###
 
 - Shorten long change-logs on test profile pages - e.g. http://openbenchmarking.org/test/pts/hpcc
 
-### POSSIBLE NEW TEST PROFILES ###
-
-- Hadoop Benchmarks / TeraSort
-- LATT Latency Tests [git://git.kernel.dk/latt.git]
-- QMC Beaver [http://qmcbeaver.sourceforge.net/]
-- Some potential interesting GPGPU tests [http://www.bealto.com/gpu-benchmarks.html]
-- WebGL / Chrome Test [http://phoronix.com/forums/showthread.php?73167-Some-Benchmarks-to-add-to-PTS-WebGL-and-Google-Chrome]
-- GtkPerf improvements [http://phoronix.com/forums/showthread.php?73047-gtkperf-I-m-maintaining-feature-requests-new-tests]
-- SIPP [http://sipp.sourceforge.net/]
-- oclGaussCrack [https://hashcat.net/oclGaussCrack/]
-- http://hashcat.net/oclhashcat/
-- Handbrake [http://handbrake.fr/]
-- libMicro [https://github.com/portante/libMicro]
-- http://markjstock.org/pages/rad_bench.html
-- https://code.google.com/p/pyrit/
-- https://github.com/acangiano/ruby-benchmark-suite
-- http://www.nwchem-sw.org/index.php/Compiling_NWChem
-- http://sourceforge.net/projects/ffsb/
-- http://www.ratgpu.com/
-- http://panthema.net/2013/pmbw/
-- http://parsec.cs.princeton.edu/parsec3-doc.htm
-- http://www.capsl.udel.edu/splash/Download.html
-- stephentu/silo
-- https://code.google.com/p/smhasher/wiki/SMHasher
-- http://benchmarksgame.alioth.debian.org/u32/performance.php?test=fasta / http://benchmarksgame.alioth.debian.org/u32/program.php?test=nbody&lang=gcc&id=1#sourcecode
-- https://openfoamwiki.net/index.php/Benchmarks_standard_v1
-- Cassandra 1.2 stress - http://docs.datastax.com/en/cassandra/1.2/cassandra/tools/toolsCStress_t.html
-- http://gcc.opensuse.org/c++bench-frescobaldi/
-- https://gooseberry.blender.org/gooseberry-production-benchmark-file/
-- https://github.com/gormanm/mmtests
-- https://github.com/soumith/convnet-benchmarks
-- http://apt.cs.manchester.ac.uk/projects/PAMELA/tools/SLAMBench/
-- https://haasn.xyz/posts/2017-10-05-how-to-benchmark-mpvs-raw-throughput.html
-
 ### WEB SOCKET / HTML5 USER INTERFACE WORK ###
 
 - Implement WebSocket security - add the password prompts from ~/.phoronix-test-suite/user-config.xml, etc
@@ -75,4 +38,4 @@ PHORONIX TEST SUITE PUBLIC TODO LIST:
 - Implement the complementary WebSocket communication to HTTP pulls
 - Ensure sane validating of more user input / resource uploads
 - Hook up TriggerTarget for allowing event.php to optionally specify a sub-set of systems/groups to target for a given trigger
-- Clean up the code and modularize more of the code into functions
+- Clean up the code and modularize more of the code

documentation/man-pages/phoronix-test-suite.1

@@ -1,4 +1,4 @@
-.TH phoronix-test-suite 1  "www.phoronix-test-suite.com" "7.8.0m3"
+.TH phoronix-test-suite 1  "www.phoronix-test-suite.com" "9.0.0m3"
 .SH NAME
 phoronix-test-suite \- The Phoronix Test Suite is an extensible open-source platform for performing testing and performance evaluation.
 .SH SYNOPSIS
@@ -6,12 +6,31 @@ phoronix-test-suite \- The Phoronix Test Suite is an extensible open-source plat
 .br
 .B phoronix-test-suite benchmark [test | suite]
 .SH DESCRIPTION
-The **Phoronix Test Suite** is the most comprehensive testing and benchmarking platform available for Linux, Solaris, OS X, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners.
+The **Phoronix Test Suite** is the most comprehensive testing and benchmarking platform available for Linux, Solaris, macOS, Windows, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners.
 .SH OPTIONS
 .TP
+.SH SYSTEM
+.B diagnostics
+This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
+.TP
+.B interactive
+A simple text-driven interactive interface to the Phoronix Test Suite.
+.TP
+.B php-conf
+This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
+.TP
+.B shell
+A simple text-driven shell interface / helper to the Phoronix Test Suite. Ideal for those that may be new to the Phoronix Test Suite
+.TP
+.B system-info
+Display the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.
+.TP
+.B system-sensors
+Display the installed system hardware and software sensors in real-time as detected by the Phoronix Test Suite Phodevi Library.
+.TP
 .SH TEST INSTALLATION
 .B force-install [Test | Suite | OpenBenchmarking ID | Test Result]  ...
-This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem.
+This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem or wishing to re-install test(s) due to compiler or other environmental changes.
 .TP
 .B install [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will install the selected test(s) inside the testing environment directory. The install process from downloading of the test files to the installation is fully automated. The install option needs to be supplied with the test name or suite as an argument. Optionally, a OpenBenchmarking.org ID or the name of a saved results file can be supplied as well and the test(s) to install will automatically be extracted from that information. If the test is already installed and was run by the latest version of the installation process, no action will be taken. Multiple arguments can be supplied to install additional tests at the same time.
@@ -50,6 +69,12 @@ This option can be used if you wish to run all of the tests found in a supplied
 .B stress-run [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will run the passed tests/suites in the multi-process stress-testing mode. The stress-run mode will not produce a result file but is rather intended for running multiple test profiles concurrently to stress / burn-in the system. The number of tests to run concurrently can be toggled via the PTS_CONCURRENT_TEST_RUNS environment variable and by default is set to a value of 2.
 .TP
+.B strict-benchmark [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option is equivalent to the `benchmark` option except it enables various options to run benchmarks an extended number of times for ensuring better statistical accuracy if enforcing strict controls over the data quality, in some cases running the benchmarks for 20+ times.
+.TP
+.B strict-run [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option is equivalent to the `run` option except it enables various options to run benchmarks an extended number of times for ensuring better statistical accuracy if enforcing strict controls over the data quality, in some cases running the benchmarks for 20+ times.
+.TP
 .SH BATCH TESTING
 .B batch-benchmark [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option and its arguments are equivalent to the benchmark option, but the process will be run in the Phoronix Test Suite batch mode.
@@ -69,6 +94,9 @@ This option will install the selected test(s) (if needed) and will proceed to ru
 .B default-run [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option will run the selected test(s). The name of the test or suite must be supplied or the OpenBenchmarking.org ID or saved local file name. Multiple arguments can be supplied to run additional tests at the same time and save the results in a suite-like fashion. Unlike the normal run option, the default-run will not prompt the user to select from the available test options but will instead use the default options as automatically set by pts-core or the test profile. Use batch-run to automatically test all of the available options.
 .TP
+.B dry-run [Test | Suite | OpenBenchmarking ID | Test Result]  ...
+This option and its arguments pre-set the Phoronix Test Suite batch run mode with enforcing of defaults to not save any results and other behavior intended for a dry/test run. This option is primarily intended for testing/evaluation purposes.
+.TP
 .B internal-run [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option and its arguments pre-set the Phoronix Test Suite batch run mode with sane values for carrying out benchmarks in a semi-automated manner and without uploading any of the result data to the public OpenBenchmarking.org.
 .TP
@@ -85,9 +113,6 @@ This option will attempt to cache the test profile/suite meta-data from OpenBenc
 .B openbenchmarking-changes
 This option will list recent changes to test profiles of enabled OpenBenchmarking.org repositories.
 .TP
-.B openbenchmarking-launcher
-This option is called automatically with the .openbenchmarking MIME file extension support for launching OpenBenchmarking.org operations.
-.TP
 .B openbenchmarking-login
 This option is used for controlling your Phoronix Test Suite client options for OpenBechmarking.org and syncing the client to your account.
 .TP
@@ -97,6 +122,12 @@ This option is used for refreshing the stored OpenBenchmarking.org repostory inf
 .B openbenchmarking-repositories
 This option will list the OpenBenchmarking.org repositories currently linked to this Phoronix Test Suite client instance.
 .TP
+.B openbenchmarking-uploads
+This option will list any recent test result uploads from the system's IP address to OpenBenchmarking.org.
+.TP
+.B recently-added-tests
+This option will list the most recently added (newest) test profiles.
+.TP
 .B upload-result [Test Result]
 This option is used for uploading a test result to OpenBenchmarking.org.
 .TP
@@ -106,35 +137,25 @@ This option can be used for uploading a test profile to your account on OpenBenc
 .B upload-test-suite
 This option can be used for uploading a test suite to your account on OpenBenchmarking.org. By uploading your test suite to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution.
 .TP
-.SH SYSTEM
-.B diagnostics
-This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
-.TP
-.B interactive
-A simple text-driven interactive interface to the Phoronix Test Suite.
-.TP
-.B php-conf
-This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.
-.TP
-.B system-info
-Display the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.
-.TP
-.B system-sensors
-Display the installed system hardware and software sensors in real-time as detected by the Phoronix Test Suite Phodevi Library.
-.TP
 .SH INFORMATION
 .B info [Test | Suite | OpenBenchmarking ID | Test Result]
 This option will show details about the supplied test, suite, virtual suite, or result file.
 .TP
+.B list-all-tests
+This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories. Unlike the other test listing options, list-all-tests will show deprecated tests, potentially broken tests, or other tests not recommended for all environments. The only check in place is ensuring the test profiles are at least compatible with the operating system in use.
+.TP
 .B list-available-suites
 This option will list all test suites that are available from the enabled OpenBenchmarking.org repositories.
 .TP
 .B list-available-tests
-This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state.
+This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state. If the system has no Internet access, it will only list the test profiles where the necesary test assets are available locally on the system or on an available network cache (the same behavior as using the list-cached-tests sub-command), unless using the list-all-tests option to override this behavior.
 .TP
 .B list-available-virtual-suites
 This option will list all available virtual test suites that can be dynamically created based upon the available tests from enabled OpenBenchmarking.org repositories.
 .TP
+.B list-cached-tests
+This option will list all test profiles where any needed test profiles are already cached or available from the local system under test. This is primarily useful if testing offline/behind-the-firewall and other use-cases where wanting to rely only upon local data.
+.TP
 .B list-installed-dependencies
 This option will list all of the packages / external test dependencies that are already installed on the system that the Phoronix Test Suite may potentially depend upon by test profiles.
 .TP
@@ -169,6 +190,9 @@ This option provides command-line searching abilities for test profiles / test s
 .B build-suite
 This option will guide the user through the process of generating their own test suite, which they can then run. Optionally, passed as arguments can be the test(s) or suite(s) to add to the suite to be created, instead of being prompted through the process.
 .TP
+.B create-test-profile
+This option can be used for creating a Phoronix Test Suite test profile by answering questions about the test for constructing the test profile XML meta-data and handling other boiler-plate basics for getting started in developing new tests.
+.TP
 .B debug-benchmark [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This option is intended for use by test profile writers and is identical to the <em>run</em> option but will yield more information during the run process that can be used to debug issues with a test profile or to verify the test profile is functioning correctly.
 .TP
@@ -184,13 +208,16 @@ This option will check all download links within the specified test profile(s) t
 .B download-test-files [Test | Suite | OpenBenchmarking ID | Test Result]  ...
 This will download the selected test file(s) to the Phoronix Test Suite download cache but will not install the tests.
 .TP
+.B inspect-test-profile [Test]
+This option can be used for inspecting a Phoronix Test Suite test profile with providing inside details on test profiles for debugging / evaluation / learning purposes.
+.TP
 .B result-file-to-suite [Test Result]
 This option will guide the user through the process of generating their own test suite, which they can then run, that is based upon an existing test results file.
 .TP
 .B validate-result-file
 This option can be used for validating a Phoronix Test Suite result file as being compliant against the OpenBenchmarking.org specification.
 .TP
-.B validate-test-profile
+.B validate-test-profile [Test]
 This option can be used for validating a Phoronix Test Suite test profile as being compliant against the OpenBenchmarking.org specification.
 .TP
 .B validate-test-suite
@@ -203,6 +230,12 @@ This option will generate a candlestick graph showing the distribution of result
 .B auto-sort-result-file [Test Result]
 This option is used if you wish to automatically attempt to sort the results by their result identifier string.
 .TP
+.B compare-results-to-baseline [Test Result] [Test Result]
+This option will allows you to specify a result as a baseline (first parameter) and a second result file (second parameter) that will offer some analysis for showing how the second result compares to the first in matching tests.
+.TP
+.B compare-results-two-way [Test Result]
+This option will allows you to specify a result file and from there to compare two individual runs within that result file for looking at wins/losses and other metrics in a head-to-head type comparison.
+.TP
 .B edit-result-file [Test Result]
 This option is used if you wish to edit the title and description of an existing result file.
 .TP
@@ -215,12 +248,15 @@ This option will manually merge multiple sets of test results generated by the P
 .B refresh-graphs [Test Result]
 This option will re-render and save all result graphs within a saved file. This option can be used when making modifications to the graphing code or its color/option configuration file and testing the changes.
 .TP
-.B remove-from-result-file [Test Result]
-This option is used if there is a set of test results you wish to remove/delete from a saved results file. The user must specify a saved results file and then they will be prompted to select the results identifier associated with the results they wish to remove.
-.TP
 .B remove-result [Test Result]
 This option will permanently remove the saved file set that is set as the first argument.
 .TP
+.B remove-results-from-result-file [Test Result]
+This option is used if there are test results (benchmarks) to be dropped from a given result file. The user must specify a saved results file and then they will be prompted to select the tests/benchmarks to remove.
+.TP
+.B remove-run-from-result-file [Test Result]
+This option is used if there is a set of test results you wish to remove/delete from a saved results file. The user must specify a saved results file and then they will be prompted to select the results identifier associated with the results they wish to remove.
+.TP
 .B rename-identifier-in-result-file [Test Result]
 This option is used if you wish to change the name of the identifier in a test results file that is shown in the Phoronix Test Suite Results Viewer and the contained graphs.
 .TP
@@ -230,6 +266,15 @@ This option is used if you wish to change the name of the saved name of a result
 .B reorder-result-file [Test Result]
 This option is used if you wish to manually change the order in which test results are shown in the Phoronix Test Suite Results Viewer and the contained graphs. The user must specify a saved results file and then they will be prompted to select the results identifiers one at a time in the order they would like them to be displayed from left to right.
 .TP
+.B result-file-confidence [Test Result]
+This option will read a saved test results file and display various statistics on the confidence of the results with the standard deviation, three-sigma values, and other metrics while color-coding "passing" results in green.
+.TP
+.B result-file-raw-to-csv [Test Result]
+This option will read a saved test results file and output the raw result file run data to a CSV file. This raw (individual) result file output is intended for data analytic purposes where the result-file-to-csv is more end-user-ready.
+.TP
+.B result-file-stats [Test Result]
+This option is used if you wish to analyze a result file by seeing various statistics on the result data for result files containing at least two sets of data.
+.TP
 .B result-file-to-csv [Test Result]
 This option will read a saved test results file and output the system hardware and software information along with the results to a CSV output. The CSV (Comma Separated Values) output can then be loaded into a spreadsheet for easy viewing.
 .TP
@@ -245,10 +290,13 @@ This option will read a saved test results file and output the system hardware a
 .B show-result [Test Result]
 Open up the test results in the Phoronix Test Suite Result Viewer or on OpenBenchmarking.org.
 .TP
-.B winners-and-losers [Test Result]
+.B wins-and-losses [Test Result]
 This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.
 .TP
 .SH OTHER
+.B commands
+This option will display a short list of possible Phoronix Test Suite commands.
+.TP
 .B debug-dependency-handler
 This option is used for testing the distribution-specific dependency handler for external dependencies.
 .TP
@@ -259,14 +307,14 @@ This option is used during the development of the Phoronix Test Suite software f
 This option is used during the development of the Phoronix Test Suite software for testing of internal interfaces, commands, and other common code-paths. The produced numbers should only be comparable for the same version of the Phoronix Test Suite, on the same hardware/software system, conducted on the same day of testing. This isn't intended as any scientific benchmark but simply to stress common PHP code-paths and looking for hot areas to optimize, etc.
 .TP
 .B help
-This option will display the list of available Phoronix Test Suite commands.
+This option will display a list of available Phoronix Test Suite commands and possible parameter types.
 .TP
 .B version
 This option will display the Phoronix Test Suite client version.
 .TP
 .SH MODULES
 .B auto-load-module
-This option can be used for easily adding a module to the LoadModules list in the Phoronix Test Suite user configuration file. That list controls what PTS modules are automatically loaded on start-up of the Phoronix Test Suite.
+This option can be used for easily adding a module to the AutoLoadModules list in the Phoronix Test Suite user configuration file. That list controls what PTS modules are automatically loaded on start-up of the Phoronix Test Suite.
 .TP
 .B list-modules
 This option will list all of the available Phoronix Test Suite modules on this system.
@@ -281,7 +329,7 @@ This option will allow you to configure all available end-user options for a Pho
 This option can be used for debugging a Phoronix Test Suite module.
 .TP
 .B unload-module
-This option can be used for easily removing a module from the LoadModules list in the Phoronix Test Suite user configuration file. That list controls what modules are automatically loaded on start-up of the Phoronix Test Suite.
+This option can be used for easily removing a module from the AutoLoadModules list in the Phoronix Test Suite user configuration file. That list controls what modules are automatically loaded on start-up of the Phoronix Test Suite.
 .TP
 .SH USER CONFIGURATION
 .B enterprise-setup
@@ -299,32 +347,26 @@ This option can be used for resetting the Phoronix Test Suite user configuration
 .B user-config-set
 This option can be used for setting an XML value in the Phoronix Test Suite user configuration file.
 .TP
-.SH WEB / GUI SUPPORT
-.B gui
-Launch the Phoronix Test Suite HTML5 web user-interface in the local GUI mode (no remote web support) and attempt to auto-launch the web-browser. THIS FEATURE IS CURRENTLY EXPERIMENTAL AND NO LONGER ACTIVE DEVELOPMENT. See Phoronix Test Suite Phoromatic as an alternative web UI approach.
-.TP
-.B start-remote-gui-server
-Start the GUI web server and WebSocket server processes for remote (or local) access via the web-browser. The settings can be configured via the Phoronix Test Suite's XML configuration file.
-.TP
-.B start-ws-server
-Manually start a WebSocket server for communication by remote Phoronix Test Suite GUIs, the Phoronix Test Suite Multi-System Commander, and other functionality. This function checks the PTS_WEBSOCKET_PORT and PTS_WEBSOCKET_SERVER environment variables for configuration.
-.TP
 .SH PHOROMATIC
+.B start-phoromatic-server
+Start the Phoromatic web server for controlling local Phoronix Test Suite client systems to facilitate automated and repeated test orchestration and other automated features targeted at the enterprise.
+.TP
+.SH RESULT VIEWER
 .B n
 _
 .TP
 .SH SEE ALSO
 .B Websites:
 .br
-http://www.phoronix-test-suite.com/
+https://www.phoronix-test-suite.com/
 .br
-http://commercial.phoronix-test-suite.com/
+https://commercial.phoronix-test-suite.com/
 .br
-http://www.openbenchmarking.org/
+https://www.openbenchmarking.org/
 .br
-http://www.phoronix.com/
+https://www.phoronix.com/
 .br
-http://www.phoronix.com/forums/
+https://www.phoronix.com/forums/
 .SH AUTHORS
-Copyright 2008 - 2018 by Phoronix Media, Michael Larabel.
+Copyright 2008 - 2019 by Phoronix Media, Michael Larabel.
 .TP

documentation/old-docs/results-parser.xml

@@ -1,111 +0,0 @@
-<?xml version="1.0"?>
-<?xml-stylesheet type="text/xsl" href="xsl/pts-definitions-viewer.xsl" ?>
-<PhoronixTestSuite>
-	<Definitions>
-		<Define>
-			<Name>P_RESULTS_PARSER_MATCH_TO_TEST_ARGUMENTS</Name>
-			<Value>PhoronixTestSuite/ResultsParser/MatchToTestArguments</Value>
-			<Description>If there are multiple ResulsParser sections within the XML file and this tag is set, the Phoronix Test Suite will see if this string appears within the PTS_TEST_ARGUMENTS in determining if this is the ResultsParser section to use for parsing the results.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_TEMPLATE</Name>
-			<Value>PhoronixTestSuite/ResultsParser/OutputTemplate</Value>
-			<Description>The section of the test output to look for the result(s) within.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_RESULT_KEY</Name>
-			<Value>PhoronixTestSuite/ResultsParser/ResultKey</Value>
-			<Description>The key to look for when parsing the OutputTemplate to find where the result should be stored. If this tag is not implemented, the default key will be used.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_LINE_HINT</Name>
-			<Value>PhoronixTestSuite/ResultsParser/LineHint</Value>
-			<Description>A unique string to search the test result output for to indicate this is the line that contains the result value.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_LINE_BEFORE_HINT</Name>
-			<Value>PhoronixTestSuite/ResultsParser/LineBeforeHint</Value>
-			<Description>The LineBeforeHint tag is similar to LineHint but is used to indicate the line before where the test results appear, in the event the LineHint tag cannot be used reliably.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_LINE_AFTER_HINT</Name>
-			<Value>PhoronixTestSuite/ResultsParser/LineAfterHint</Value>
-			<Description>The LineAfterHint tag is similar to LineBeforeHint but is used to indicate the line after where the test results appear, in the event the LineHint or LineBeforeHint tags cannot be used reliably.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_RESULT_BEFORE_STRING</Name>
-			<Value>PhoronixTestSuite/ResultsParser/ResultBeforeString</Value>
-			<Description>The ResultBeforeString tag is optional and can be used if there is a consistient string following the reporting of the result, in the event the result's location is not static within the line string.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_STRIP_FROM_RESULT</Name>
-			<Value>PhoronixTestSuite/ResultsParser/StripFromResult</Value>
-			<Description>Any string that should be searched for and removed from the result value anywhere within the value string.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_STRIP_RESULT_POSTFIX</Name>
-			<Value>PhoronixTestSuite/ResultsParser/StripResultPostfix</Value>
-			<Description>This is similar to the StripFromResult tag but will only look at the end of the value string for this string to be removed. (I.e. if a unit such as 'ms' or 'FPS' is appended to the result by the test without a space.)</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_MULTI_MATCH</Name>
-			<Value>PhoronixTestSuite/ResultsParser/MultiMatch</Value>
-			<Description>If the result value should be found multiple times within the result file and then some mathematical operation carried out, it should be set using this tag. Setting the value to AVERAGE will find all matches and then calculate the average value based upon matches. While not required, setting the value to NONE will use the default code-path of only looking for one result value.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_DIVIDE_BY</Name>
-			<Value>PhoronixTestSuite/ResultsParser/DivideResultBy</Value>
-			<Description>If the result value should be divided by some constant (to convert the result to a different unit), the value should be supplied in this tag.</Description>
-		</Define>
-		<Define>
-			<Name>P_RESULTS_PARSER_MULTIPLY_BY</Name>
-			<Value>PhoronixTestSuite/ResultsParser/MultiplyResultBy</Value>
-			<Description>If the result value should be multiplied by some constant (to convert the result to a different unit), the value should be supplied in this tag.</Description>
-		</Define>
-		<Define>
-			<Name>P_IMAGE_PARSER_MATCH_TO_TEST_ARGUMENTS</Name>
-			<Value>PhoronixTestSuite/ImageParser/MatchToTestArguments</Value>
-			<Description>If there are multiple ResulsParser sections within the XML file and this tag is set, the Phoronix Test Suite will see if this string appears within the PTS_TEST_ARGUMENTS in determining if this is the ResultsParser section to use for parsing the results.</Description>
-		</Define>
-		<Define>
-			<Name>P_IMAGE_PARSER_SOURCE_IMAGE</Name>
-			<Value>PhoronixTestSuite/ImageParser/SourceImage</Value>
-			<Description>If the test profile is for an image quality comparison, the ImageFile tag should contain the file/path of the source file.</Description>
-		</Define>
-		<Define>
-			<Name>P_IMAGE_PARSER_IMAGE_X</Name>
-			<Value>PhoronixTestSuite/ImageParser/ImageX</Value>
-			<Description>If the test profile is for an image quality comparison, the ImageX should be the X coordinate of where to begin cropping the source file.</Description>
-		</Define>
-		<Define>
-			<Name>P_IMAGE_PARSER_IMAGE_Y</Name>
-			<Value>PhoronixTestSuite/ImageParser/ImageY</Value>
-			<Description>If the test profile is for an image quality comparison, the ImageX should be the Y coordinate of where to begin cropping the source file.</Description>
-		</Define>
-		<Define>
-			<Name>P_IMAGE_PARSER_IMAGE_WIDTH</Name>
-			<Value>PhoronixTestSuite/ImageParser/ImageWidth</Value>
-			<Description>If the test profile is for an image quality comparison, the ImageWidth should be the produced image width.</Description>
-		</Define>
-		<Define>
-			<Name>P_IMAGE_PARSER_IMAGE_HEIGHT</Name>
-			<Value>PhoronixTestSuite/ImageParser/ImageHeight</Value>
-			<Description>If the test profile is for an image quality comparison, the ImageHeight should be the produced image height.</Description>
-		</Define>
-		<Define>
-			<Name>P_MONITOR_PARSER_SENSOR</Name>
-			<Value>PhoronixTestSuite/SystemMonitor/Sensor</Value>
-			<Description>If the test profile is to monitor any system sensors during the testing process, the name of the sensor (via the Phodevi name; i.e. cpu.usage) should be supplied. While not a Phodevi sensor, if passing sys.time as the sensor value, the time it takes to execute the test will be recorded.</Description>
-		</Define>
-		<Define>
-			<Name>P_MONITOR_PARSER_FREQUENCY</Name>
-			<Value>PhoronixTestSuite/SystemMonitor/PollingFrequency</Value>
-			<Description>If the test profile is to monitor any system sensors during the testing process, how often the matched sensor is to be polled (in seconds) should be supplied. Floating values are supported.</Description>
-		</Define>
-		<Define>
-			<Name>P_MONITOR_PARSER_REPORT</Name>
-			<Value>PhoronixTestSuite/SystemMonitor/Report</Value>
-			<Description>If the test profile is to monitor any system sensors during the testing process, this tag provides how the monitored results should be reported. Supported values include MAX, MIN, AVG, and ALL.</Description>
-		</Define>
-	</Definitions>
-</PhoronixTestSuite>

documentation/old-docs/test-profile-downloads.xml

@@ -1,42 +0,0 @@
-<?xml version="1.0"?>
-<?xml-stylesheet type="text/xsl" href="xsl/pts-definitions-viewer.xsl" ?>
-<PhoronixTestSuite>
-	<Definitions>
-		<Define>
-			<Name>P_DOWNLOADS_PACKAGE_URL</Name>
-			<Value>PhoronixTestSuite/Downloads/Package/URL</Value>
-			<Description>The URL(s) to download the package from. If multiple URLs are supplied (delimited by a comma), the Phoronix Test Suite will randomly choose one URL but will fallback to other URLs if one fails either due to the file missing, the MD5 changed, or server inaccessability.</Description>
-			<Required>YES</Required>
-		</Define>
-		<Define>
-			<Name>P_DOWNLOADS_PACKAGE_MD5</Name>
-			<Value>PhoronixTestSuite/Downloads/Package/MD5</Value>
-			<Description>The MD5 check-sum for the file to be downloaded. This is to ensure file integrity. If no string is supplied, no MD5 check will be performed.</Description>
-			<Required>RECOMMENDED</Required>
-		</Define>
-		<Define>
-			<Name>P_DOWNLOADS_PACKAGE_FILENAME</Name>
-			<Value>PhoronixTestSuite/Downloads/Package/FileName</Value>
-			<Description>If this tag is set, once the file is downloaded it will be renamed to this value. If this tag is not set, the remote file-name will be used.</Description>
-			<Required>NO</Required>
-		</Define>
-		<Define>
-			<Name>P_DOWNLOADS_PACKAGE_FILESIZE</Name>
-			<Value>PhoronixTestSuite/Downloads/Package/FileSize</Value>
-			<Description>The size of the file to be downloaded should be the value of this tag. The FileSize should be an integer that is the number of bytes that makes up the file. This tag is used so that the Phoronix Test Suite can determine the total size of files that need to be downloaded for a given operation.</Description>
-			<Required>RECOMMENDED</Required>
-		</Define>
-		<Define>
-			<Name>P_DOWNLOADS_PACKAGE_PLATFORMSPECIFIC</Name>
-			<Value>PhoronixTestSuite/Downloads/Package/PlatformSpecific</Value>
-			<Description>If this package is just needed on select operating systems / platforms, list them inside this tag and multiple platforms can be delimited by a comma.</Description>
-			<Required>NO</Required>
-		</Define>
-		<Define>
-			<Name>P_DOWNLOADS_PACKAGE_ARCHSPECIFIC</Name>
-			<Value>PhoronixTestSuite/Downloads/Package/ArchitectureSpecific</Value>
-			<Description>If this package is just needed on select system/processor architectures, list them inside this tag and multiple architectures can be delimited by a comma.</Description>
-			<Required>NO</Required>
-		</Define>
-	</Definitions>
-</PhoronixTestSuite>

documentation/old-docs/writing_your_first_module.html

@@ -1,73 +0,0 @@
-<html xmlns="http://www.w3.org/1999/xhtml">
-<head>
-<Title>Phoronix Test Suite - Module Writing</Title>
-<meta http-equiv="content-type" content="text/html; charset=utf-8" />
-<link href="includes/pts-documentation.css" rel="stylesheet" type="text/css" />
-</head>
-<body>
-<div id="pts_doc_header"><div class="pts_doc_fixed"><a href="http://www.phoronix-test-suite.com/"><div id="pts_doc_logo"></div></a></div></div>
-<div class="pts_doc_fixed">
-<div class="pts_doc_notice"><div style="float: left"><a href="index.html">&lt;&lt; Documentation Home</a></div><div style="float: right;">Module Writing</div></div>
-<div class="pts_doc_main">
-<!-- PTS AREA -->
-<h1>Phoronix Test Suite Modules</h1>
-<div style="width: 1px; height: 20px;"></div>
-<p>Writing a module for the Phoronix Test Suite allows new functionality to be added 
-without having to extensively learn how pts-core functions. The module framework 
-for the Phoronix Test Suite allows modules to be written as a PHP object. Example PTS modules could include a module to shutdown the 
-screensaver when the Phoronix Test Suite starts up and re-enabling it when the 
-test is over, using sendmail to forward the results to an e-mail address when 
-testing is completed, or writing the current test status to a LCDproc-enabled 
-VFD display.</p>
-<p>Modules are stored in <em>pts-core/modules/</em>. Loading a 
-module is done by either setting the <em>PTS_MODULES</em> environmental variable 
-with the name of the module (excluding the <em>.php</em> file 
-extension) or by associating a module with a separate environmental variable. The default list of modules to be loaded is stored in <em>~/.phoronix-test-suite/user-config.xml</em>.</p>
-<p>Note: To run through all of the function calls for a module without needing to run a test, 
-run <em>phoronix-test-suite test-module MODULE_NAME</em>. Additionally, running 
-<em>phoronix-test-suite debug-module MODULE_NAME</em> will yield additional debugging details while 
-executing the same process.</p>
-<h1>Module</h1>
-<p>To see all of the functions supported for modules written in PHP, look at <em>pts-core/modules/dummy_module.php</em> 
-and additionally the other .php modules that ship with the Phoronix Test Suite. 
-Additionally, there are several functions written specifically for Phoronix Test 
-Suite modules that make it easier to save files, read files, and provided multi-threading 
-support for modules. The <em>pts_timed_function()</em> makes it possible (and 
-very easy) to thread functions within a module so that at a set interval the defined 
-functions will be called. For example, this support is used heavily within the 
-<em>system_monitor</em> module to poll sensors every X seconds even while there 
-are tests running. These functions can be found within <em>pts-core/objects/pts_module.php</em>.</p>
-<p>Below is a sample module that times how long it takes to run the Phoronix Test 
-Suite. It would be saved as <em>pts-core/modules/time_me.php</em>.</p>
-<blockquote>&lt;?php<br />
-class time_me extends pts_module_interface<br />
-{<br />
- &nbsp; &nbsp; const module_name = "Time Me!";<br />
- &nbsp; &nbsp; const module_version = "1.0.0";<br />
- &nbsp; &nbsp; const module_description = "This is a module that times how long the Phoronix Test Suite runs.";<br />
- &nbsp; &nbsp; const module_author = "Phoronix Media";<br />
-<br />
- &nbsp; &nbsp; static $start_time = NULL;<br />
- &nbsp; &nbsp; static $end_time = NULL;<br />
-<br />
- &nbsp; &nbsp; public static function __startup()<br />
- &nbsp; &nbsp; {<br />
- &nbsp; &nbsp;  &nbsp; &nbsp; self::$start_time = time();<br />
- &nbsp; &nbsp; }<br />
- &nbsp; &nbsp; public static function __shutdown()<br />
- &nbsp; &nbsp; {<br />
- &nbsp; &nbsp;  &nbsp; &nbsp; self::$end_time = time();<br />
-<br />
- &nbsp; &nbsp;  &nbsp; &nbsp; $time_elapsed = self::$end_time - self::$start_time;<br />
-<br />
- &nbsp; &nbsp;  &nbsp; &nbsp; echo "\nThe Phoronix Test Suite Ran For " . $time_elapsed . " Seconds.\n";<br />
- &nbsp; &nbsp; }<br />
-}<br />
-?&gt;</blockquote>
-<p>Then by running <em>PTS_MODULES=time_me phoronix-test-suite benchmark video-extensions</em>, 
-at the end of the test it would print a string similar to: &quot;The Phoronix 
-Test Suite Ran For 52 Seconds.&quot;</p>
-<!-- END OF PTS AREA -->
-</div></div><div class="pts_doc_fixed"><div class="pts_doc_bottom"><div style="float: left;"><a href="http://www.phoronix-test-suite.com/">Phoronix-Test-Suite.com</a></div><div style="float: right;">Copyright &copy; 2008 by <a href="http://www.phoronix-media.com/">Phoronix Media</a>.</div></div></div>
-</body>
-</html>

documentation/old-docs/writing_your_first_test.html

@@ -1,118 +0,0 @@
-<html xmlns="http://www.w3.org/1999/xhtml">
-<head>
-<Title>Phoronix Test Suite - A Quick Guide To Writing A Test Profile</Title>
-<link href="includes/pts-documentation.css" rel="stylesheet" type="text/css" />
-</head>
-<body>
-<div id="pts_doc_header"><div class="pts_doc_fixed"><a href="http://www.phoronix-test-suite.com/"><div id="pts_doc_logo"></div></a></div></div>
-<div class="pts_doc_fixed">
-<div class="pts_doc_notice"><div style="float: left"><a href="index.html">&lt;&lt; Documentation Home</a></div><div style="float: right;">Writing A Test Profile</div></div>
-<div class="pts_doc_main">
-<!-- PTS AREA -->
-<h1>Writing A Test Profile</h1>
-<p>Writing a test profile for the Phoronix Test Suite is a relatively quick and easy 
-process for anyone familiar with common Linux commands and the basics of XML. 
-To help you understand the design of the Phoronix Test Suite, this guide covers 
-the steps needed to write a testing profile for a very simple application.</p>
-<p>The first step in the profile writing process is to, well, have a piece of 
-software you'd like to use with the Phoronix Test Suite. This software can be 
-closed-source or open-source and be virtually anything as long as it is compatible 
-with the Linux operating system.</p>
-<p>For this guide, the piece of software being used for demonstration is just 
-a simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz 
-formula. Below is this sample piece of software intended just for demonstration 
-purposes.</p>
-<blockquote>#include &lt;iostream&gt;<br>
-#include &lt;math.h&gt;</p>
-<p>int main()<br>
-{<br>
-double pi = 0;</p>
-<p> for(long int i = 1; i &lt;= 87654321; i++)<br>
-pi += (double) pow(-1, i + 1) / (2 * i - 1);</p>
-<p> pi *= 4;<br>
-std::cout &lt;&lt; &quot;Done Calculating Pi...&quot; &lt;&lt; endl;<br>
-return 0;<br>
-}</blockquote>
-<p>The first step in the actual profile writing process is to name it. If you're 
-looking to ultimately push this profile to be included in the Phoronix Test Suite, 
-its name must be all lower case and consist of just alpha-numeric characters, 
-but can contain dashes (-). A more advanced test profile capability is operating system prefixes, and if using those there is an 
-underscore separating the prefix from the normal profile name. For this sample profile, we're calling it <em>sample-program</em> 
-and the file-name would be <em>sample-program.xml</em>. Our (very basic) profile is showcased 
-below.</p>
-<blockquote>&lt;PhoronixTestSuite&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestProfile&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Version&gt;1.1.0&lt;/Version&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestType&gt;Processor&lt;/TestType&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;SoftwareType&gt;Utility&lt;/SoftwareType&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;License&gt;FREE&lt;/License&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Status&gt;PRIVATE&lt;/Status&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Maintainer&gt;Phoronix Media&lt;/Maintainer&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/TestProfile&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestInformation&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Title&gt;Sample Pi Program&lt;/Title&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TimesToRun&gt;3&lt;/TimesToRun&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ResultScale&gt;Seconds&lt;/ResultScale&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Proportion&gt;LIB&lt;/Proportion&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Description&gt;A simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz formula. This test can be used for showcasing how to write a basic test profile.&lt;/Description&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ExternalDependencies&gt;build-utilities&lt;/ExternalDependencies&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/TestInformation&gt;<br>
-&lt;/PhoronixTestSuite&gt;</blockquote>
-<p>This XML profile is what interfaces with the Phoronix Test Suite and provides 
-all the needed information about the test as well as other attributes. For a complete 
-listing of all the supported profile options, look at the specification files 
-in the documentation folder. In the case of <em>sample-program</em>, it lets the Phoronix 
-Test Suite know that it's composed of free software, is designed to test the processor, 
-is intended for private use only, and this profile is maintained by Phoronix Media. 
-In addition, it tells the Phoronix Test Suite to execute this program three times 
-and as no result quantifier is set, the average of the three runs will be taken. 
-This profile also tells the Phoronix Test Suite that the generic <em>build-utilities</em> 
-package is needed, which will attempt to ensure that GCC and the standard development 
-utilities/libraries are installed on your Linux distribution. This is needed as 
-the C++ source-code will need to be built from source.</p>
-<p>The next step is to write the <em>install.sh</em> file, which once called by the Phoronix 
-Test Suite is intended to install the test locally for benchmarking purposes. 
-The <em>install.sh</em> file is technically optional, but is generally used by all tests. Note: The first argument supplied to 
-the install script is the directory that the test needs to be installed to. The 
-<em>install.sh</em> file (in our instance) is to be placed inside <em>pts/test-resources/sample-program</em>. 
-Below is the <em>install.sh</em> for the <em>sample-program</em>.</p>
-<blockquote>#!/bin/sh<br /><br />
-tar -xjf sample-pi-program-1.tar.bz2<br />
-g++ sample-pi-program.cpp -o sample-pi-program<br />
-echo &quot;#!/bin/sh<br />
-\$TIMER_START<br />
-./sample-pi-program 2&gt;&amp;1<br />
-\$TIMER_STOP<br />
-&quot; &gt; sample-program<br>
-chmod +x sample-program</blockquote>
-<p>Phoronix Test Suite 1.4 and later provides an integrated multi-platform micro-timer framework that provides the <em>$TIMER_START</em> and <em>$TIMER_STOP</em> functionality.</p>
-<p>This install file builds the code with GCC, and then creates a small script 
-that is run by the Phoronix Test Suite, which times how long it takes to run the 
-software. Where does the source-code come into play? Well, it needs to be downloaded 
-now from a web server. The Phoronix Test Suite has built-in support for managing 
-downloads from multiple servers in a random over, fall-back support if one mirror 
-is done, and verification of MD5 check-sums. Below is the <em>downloads.xml</em> file for 
-<em>sample-program</em> that covers all of this.</p>
-<blockquote>&lt;PhoronixTestSuite&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Downloads&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Package&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;URL&gt;http://www.phoronix-test-suite.com/benchmark-files/sample-pi-program.cpp&lt;/URL&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;MD5&gt;e90fb790df8d1544696a1439c9b5bd8d&lt;/MD5&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/Package&gt;<br>
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/Downloads&gt;<br>
-&lt;/PhoronixTestSuite&gt;</blockquote>
-<p>The final step in the profile writing process is to write a parser to strip 
-all information but the reported result from the standard output. The standard 
-output is submitted to <em>parse-results.sh</em> or <em>parse-results.php</em> as the first argument 
-in quotes or using <em>$LOG_FILE</em> if the test profile writes to that variable's location.</p>
-<p>If the test profile uses the integrated micro-timer framework with <em>$TIMER_START</em> and <em>$TIMER_STOP</em> but no parse-results file is provided, the Phoronix Test Suite will automatically use the difference between <em>$TIMER_START</em> and <em>$TIMER_STOP</em> and use that as the test result. This is the time (in seconds) that elapsed between starting and stopping the timer.</p>
-<p>After that, with all the files in their correct locations, just run: <em>phoronix-test-suite 
-benchmark sample-program</em>. The Phoronix Test Suite should now handle the rest by 
-installing the test, running the test, and recording the results (if you so choose). 
-There is no additional work that needs to be done for the results to be recorded 
-in the PTS Results Viewer or even reporting the results to OpenBenchmarking.org.</p>
-<p><strong>For more information, visit <a href="http://www.phoronix-test-suite.com/">Phoronix-Test-Suite.com</a> and read the <a href="index.html">included documentation</a>. It should also be of great help to just look at the existing test profiles, which can be found inside the <em>phoronix-test-suite/pts/</em> folder with the <em>test-resources</em> and <em>test-profiles</em> sub-directories.</strong></p>
-<!-- END OF PTS AREA -->
-</div></div><div class="pts_doc_fixed"><div class="pts_doc_bottom"><div style="float: left;"><a href="http://www.phoronix-test-suite.com/">Phoronix-Test-Suite.com</a></div><div style="float: right;">Copyright &copy; 2008 by <a href="http://www.phoronix-media.com/">Phoronix Media</a>.</div></div></div>
-</body>
-</html>

documentation/phoromatic.html

@@ -8,7 +8,7 @@
 <p>Phoromatic is a remote management system for the Phoronix Test Suite. Phoromatic allows the automatic (hence the name <em>Phoro-matic</em>) scheduling of tests, remote installation of new tests, and the management of multiple test systems all through an intuitive, easy-to-use web interface. Tests can be scheduled to automatically run on a routine basis across multiple test systems. The test results are then available from this central, secure location.</p>
 <p>Phoromatic was originally introduced with Phoronix Test Suite 2.0 via Phoromatic.com as a project going back to 2008~2009. Phoromatic.com debuted as a hosted instance with the option of behind-the-firewall licensing for use within organizations. With Phoronix Test Suite 5.2 the model shifted to offer a local, open-source version of Phoromatic built into the Phoronix Test Suite code-base. Thanks to continued enterprise development, with Phoronix Test Suite 5.4 is now a fully-functioning, built-in version of Phoromatic that's open-source and can be used for behind-the-firewall testing without needing to push results to OpenBenchmarking.org and the ability to keep all results private.</p>
 <p>Phoromatic in Phoronix Test Suite 5.4 also has the ability to support zero-conf  network discovery using Avahi and the automatic distribution of needed test profiles/suites and test files. Phoronix Test Suite 5.4's Phoromatic is a significant breakthrough for open-source testing particularly those running this GPL benchmarking software within test labs and other large organizations.</p>
-<h2>Features<h2>
+<h2>Features</h2>
 <p>Built atop the Phoronix Test Suite, Phoromatic offers many features for both enterprise and community/personal users:</p>
 <h3>Automated Scheduling</h3>
 <p>Whether it is every evening at 6:00PM, once every Thursday at 10:00AM or somewhere in between, Phoromatic can schedule tests to be run at user-defined intervals. The testing schedules can be updated through Phoromatic web interface. After the test(s) have run, the results will be immediately uploaded to Phoromatic.</p>
@@ -44,7 +44,7 @@
 <p>This command will simply download all of the needed test files for the tests/suites passed to the sub-command. Alternatively you could also pass pts/all to cache all tests. It's important though to just cache the tests/suites you'll be using on your network. This will generate the test file download cache by default to <em>~/.phoronix-test-suite/download-cache/</em> or <em>/usr/share/phoronix-test-suite/download-cache/</em> depending upon your write permissions. You can always run this command later with more test files. Alternatively, if you already have a number of tests installed on the system, simply running "phoronix-test-suite make-download-cache" will generate the cache based upon the currently installed tests.</p>
 <p><strong>$ phoronix-test-suite make-openbenchmarking-cache</strong></p>
 <p>This command will cache as much of the OpenBenchmarking.org meta-data as possible for test profiles and test suites. After the above commands, the Phoromatic Server should no longer need a persistent Internet connection.</p>
-<p><strong>$ phororonix-test-suite start-phoromatic-server</strong></p>
+<p><strong>$ phoronix-test-suite start-phoromatic-server</strong></p>
 <p>Proceed to start the Phoromatic Server and operate as normal.</p>
 <p>For the test clients without an Internet connection, as long as they're able to reach the Phoromatic Server, the Phoromatic Server should be able to automatically serve all of the needed test files download cache and OpenBenchmarking.org meta-data to the systems locally.</p>
 <h3>Use Case C: Phoromatic Across The Internet</h3>

documentation/phoromatic.md

@@ -0,0 +1,139 @@
+
+# Phoronix Test Suite Phoromatic
+
+## Phoromatic Server
+
+### Introduction
+Phoromatic is a remote management system for the Phoronix Test Suite. Phoromatic allows the automatic (hence the name *Phoro-matic* ) scheduling of tests, remote installation of new tests, and the management of multiple test systems all through an intuitive, easy-to-use web interface. Tests can be scheduled to automatically run on a routine basis across multiple test systems. The test results are then available from this central, secure location.
+
+Phoromatic was originally introduced with Phoronix Test Suite 2.0 via Phoromatic.com as a project going back to 2008~2009. Phoromatic.com debuted as a hosted instance with the option of behind-the-firewall licensing for use within organizations. With Phoronix Test Suite 5.2 the model shifted to offer a local, open-source version of Phoromatic built into the Phoronix Test Suite code-base. Thanks to continued enterprise development, with Phoronix Test Suite 5.4 is now a fully-functioning, built-in version of Phoromatic that's open-source and can be used for behind-the-firewall testing without needing to push results to OpenBenchmarking.org and the ability to keep all results private.
+
+Phoromatic in Phoronix Test Suite 5.4 also has the ability to support zero-conf  network discovery using Avahi and the automatic distribution of needed test profiles/suites and test files. Phoronix Test Suite 5.4's Phoromatic is a significant breakthrough for open-source testing particularly those running this GPL benchmarking software within test labs and other large organizations.
+
+
+### Features
+Built atop the Phoronix Test Suite, Phoromatic offers many features for both enterprise and community/personal users:
+
+#### Automated Scheduling
+Whether it is every evening at 6:00PM, once every Thursday at 10:00AM or somewhere in between, Phoromatic can schedule tests to be run at user-defined intervals. The testing schedules can be updated through Phoromatic web interface. After the test(s) have run, the results will be immediately uploaded to Phoromatic.
+
+#### Extensible
+Any test profile or test suite that is compliant with the Phoronix Test Suite specification will work with Phoromatic. Phoromatic is able to leverage the hundreds of test profiles and test suites currently in the Phoronix Test Suite via OpenBenchmarking.org, along with any custom or proprietary test profiles you or your company utilize. Additionally, the Phoromatic interface allows the user to construct their own test suite(s).
+
+#### Remote Testing
+Once the test system is setup, all testing and management of that system can be done remotely. There is no need to execute Phoronix Test Suite commands locally using the GUI or command line version, but instead nearly all of the same features are accessible from the Phoromatic interface.
+
+#### Multi-System Support
+A single Phoromatic account is able to manage multiple test systems running the Phoronix Test Suite. Phoromatic supports grouping together test systems, tagging, and other features to support effectively managing many test systems. From the Phoromatic interface, installed system hardware and software from a given system can also be viewed.
+
+#### Turn-Key Deployment
+No additional software needs to be installed to support Phoromatic; all that's needed is Phoronix Test Suite 5.4 or later for full compatibility. New test systems can easily be synced with a given Phoromatic account by running a single command from the Phoronix Test Suite client.
+
+#### Result Management
+Test results are automatically uploaded to the Phoromatic account and remain private unless you opt to upload them to OpenBenchmarking.org. From the Phoromatic interface, results from multiple test systems can easily be compared and multiple results from the same systems can be used to track performance over time. There are also options to look at the statistical significance of the results and other features to efficiently and effectively analyze the system's performance.
+
+#### Decentralized
+Once the Phoronix Test Suite running on the Phoromatic Server has been able to cache all of the OpenBenchmarking.org test files and the needed files for each test, Phoromatic with any Phoronix Test Suite clients on your LAN can run fully decentralized without the need for a constant stream of OpenBenchmarking.org communication or Internet connection for that matter. (The only exception would be if your local systems don't have all their needed external dependencies and your system's package manager would need to install components like a compiler or necessary system libraries.
+
+#### Fully Open-Source
+Phoromatic is now fully open-source within the Phoronix Test Suite code-base for fostering greater development and new capabilities. Patches are welcome and Phoronix Media is available to provide commercial support and custom engineering services around Phoromatic and the Phoronix Test Suite.
+
+
+### Phoromatic Server Setup
+Phoromatic is built into the Phoronix Test Suite code-base and should be found in all packaged versions of the **phoronix-test-suite** . Starting the Phoromatic Server entails running phoronix-test-suite start-phoromatic-server after configuring the server information within *~/.phoronix-test-suite/user-config.xml* . The Phoromatic Server can with or without root permissions depending upon your firewall and the port numbers you wish to use for the server.
+
+On the "client side", any up-to-date version of the Phoronix Test Suite can automatically communicate with the Phoromatic Server. If Avahi support is available (commonly in Linux distribution repositories as _avahi-tools_ ), there should be zero-conf discovery if the Phoromatic Server and client systems are on the same LAN. If a Phoronix Test Suite client discovers a Phoromatic Server, it will attempt to use it automatically as a local download cache. In the event of no Internet connection, it will also attempt to obtain the needed OpenBenchmarking.org test/suite meta-data from the Phoromatic Server based upon its archived meta-data. This allows the Phoronix Test Suite / Phoromatic deployment on the LAN to be self-sustaining without an Internet connection as long as the systems have all installed test dependencies.
+
+Further configuration of the setup parameters for the Phoromatic Server and Phoronix Test Suite clients can be tuned via the *~/.phoronix-test-suite/user-config.xml* file. All control and configuration of the Phoromatic Server is done via the web-based interface when the Phoromatic Server is active.
+
+The Phoromatic Server utilizes PHP/HHVM's built-in web-server capabilities and there's also a Phoronix Test Suite built-in WebSocket server that's also initiated for back-end processing. At this time there are no ports set by default for these services but must be defined within the user configuration file. With the Avahi zero-conf network discovery and other automated detection in place, there's little restrictions over the port selection.
+
+Systemd and Upstart service files are shipped with the Phoronix Test Suite for those that wish to have the services automatically run as daemons. The only new requirements over the basic Phoronix Test Suite system requirements is having PHP-SQLite support installed and the newer version of PHP or HHVM is recommended for offering the best support.
+
+
+### Example Deployments
+#### Use Case A: Unrestricted Internet Access, Local Result Storage
+Systems on your network with unrestricted Internet access is the easiest and simplest deployment for the Phoronix Test Suite and Phoromatic. After installing the Phoronix Test Suite on the system you wish to designate the Phoromatic Server and have configured the *user-config.xml* file, simply run:
+
+**$ phoronix-test-suite start-phoromatic-server**
+
+Assuming you have no firewall or permission issues, the built-in web server and WebSocket server should proceed to initiate along with outputting the IP/port information for these services. Unless otherwise disabled from the user configuration file and if avahi-tools is present, the Phoromatic Server will be advertised with Avahi for zero-configuration networking.
+
+From the Phoromatic web interface you are able to create an account and from there proceed with the creating of test schedules, updating settings, and connecting systems. From the "client systems" you wish to use as the benchmarking nodes, it's simply a matter of running **phoronix-test-suite phoromatic.connect** with zero-conf networking or otherwise follow the information from the Phoromatic web interface for manual setup with the IP/port information.
+
+#### Use Case B: No Internet Available To Client Systems
+It's possible to run the Phoronix Test Suite and Phoromatic Server without a persistent Internet connection as long as you are able to first download the necessary files to the Phoromatic Server. After installing the Phoronix Test Suite on the system you wish to designate the Phoromatic Server and have configured the *user-config.xml* file, a few commands from the system while having an Internet connection will be able to cache the needed data:
+
+**$ phoronix-test-suite make-download-cache x264 xonotic ffmpeg**
+
+This command will simply download all of the needed test files for the tests/suites passed to the sub-command. Alternatively you could also pass pts/all to cache all tests. It's important though to just cache the tests/suites you'll be using on your network. This will generate the test file download cache by default to *~/.phoronix-test-suite/download-cache/* or */usr/share/phoronix-test-suite/download-cache/* depending upon your write permissions. You can always run this command later with more test files. Alternatively, if you already have a number of tests installed on the system, simply running "phoronix-test-suite make-download-cache" will generate the cache based upon the currently installed tests.
+
+**$ phoronix-test-suite make-openbenchmarking-cache**
+
+This command will cache as much of the OpenBenchmarking.org meta-data as possible for test profiles and test suites. After the above commands, the Phoromatic Server should no longer need a persistent Internet connection.
+
+**$ phoronix-test-suite start-phoromatic-server**
+
+Proceed to start the Phoromatic Server and operate as normal.
+
+For the test clients without an Internet connection, as long as they're able to reach the Phoromatic Server, the Phoromatic Server should be able to automatically serve all of the needed test files download cache and OpenBenchmarking.org meta-data to the systems locally.
+
+#### Use Case C: Phoromatic Across The Internet
+If wishing to use the same Phoromatic Server across multiple geographic locations, it's easily possible -- you just lose out on the zero-conf networking ability. To let the Phoronix Test Suite client systems know about the remote Phoromatic Server, simply add the Phoromatic Server information to the client's *PhoromaticServers* element within the *user-config.xml* . Of course, make sure the Phoromatic Server has a globally resolvable IP address and its Phoromatic HTTP/WebSocket ports are open. Once informing the client of the Phoromatic Server, the use cases as above apply in the same manner.
+
+
+### Client Setup
+From Phoronix Test Suite client systems running on the LAN, the following command will report all available detected Phoromatic Servers along with important server and debugging information:
+
+**$ phoronix-test-suite phoromatic.explore**
+
+With the following example output on finding one successful server:
+
+*IP: 192.168.1.211
+HTTP PORT: 5447
+WEBSOCKET PORT: 5427
+SERVER: PHP 5.5.9-1ubuntu4.4 Development Server
+PHORONIX TEST SUITE: Phoronix Test Suite v5.4.0m1 [5313]
+DOWNLOAD CACHE: 19 FILES / 2390 MB CACHE SIZE
+SUPPORTED OPENBENCHMARKING.ORG REPOSITORIES:
+      pts - Last Generated: 05 Oct 2014 07:16*
+
+Phoromatic Servers are detected by the Phoronix Test Suite through Avahi or if manually configuring the Phoronix Test Suite clients to point to Phoromatic Servers. For networks without Avahi/auto-discovery support or for test systems that may be connecting from another network, the IP address and HTTP port number can be added to the local system's *~/.phoronix-test-suite/user-config.xml* with the *PhoromaticServers* element. Adding the *IP:port* (the Phoromatic Server's HTTP port) to the PhoromaticServers *user-config.xml* element for will perform targeted probing by the Phoronix Test Suite without any dependence on Avahi. Multiple Phoromatic Servers can be added if each IP:port is delimited by a comma.
+
+To connect a Phoronix Test Suite system for benchmarking to an account, log into your Phoromatic account from the web-interface and on the main/system pages will be instructions along with a specially formed string to run, e.g. *phoronix-test-suite phoromatic.connect 192.168.1.211:5447/I0SSJY* . When running that command once on the system(s) to be synced to that account, as the administrator you'll be able to validate/approve the systems from the Phoromatic web interface. After that, whenever the system(s) are to be running benchmarks, simply have the **phoronix-test-suite phoromatic.connect** command running on the system (after the initial account has been synced, simply running **phoronix-test-suite phoromatic.connect** is enough for the system to find the server and its account).
+
+
+### Root Administrator
+The root administrator account is able to manage the server-level settings, e.g. Phoromatic storage location and other global settings related to the Phoronix Test Suite / Phoromatic Server, from the web user-interface.
+
+To enable the root administrator log-in, first from the server's command-line interface run **phoronix-test-suite phoromatic.set-root-admin-password** to set the password. Following that, you can log into the root administrator account via the web interface via the *rootadmin* user-name and the set password.
+
+
+### Other Advice
+#### Disable Internet Precaution
+If you have an Internet connection but want to ensure your Phoronix Test Suite client doesn't attempt to use it for any matter, via the *~/.phoronix-test-suite/user-config.xml* you can set *NoInternetCommunication* to *TRUE* . There's also a NoNetworkCommunication tag, but setting that to TRUE will disable any form of network communication -- including communication with the Phoromatic Server.
+
+#### Ports / Services
+The Phoromatic Server process currently relies upon a PHP/HHVM built-in web server process and a PTS-hosted WebSocket server. The web server process handles the web UI and much of the responsibilities of the Phoromatic Server. Over time the PTS WebSocket server will be increasingly utilized for bi-directional, real-time communication between the server and clients -- including for features like viewing real-time hardware sensors of client systems from the server UI.
+
+#### Systemd / Upstart
+Packaged with the Phoronix Test Suite are basic *phoromatic-client* and *phoromatic-server* configurations for both Upstart and systemd init systems. The *phoromatic-server* configuration will launch the Phoronix Test Suite's Phoromatic Server and the *phoromatic-client* service will attempt to connect to a _pre-configured_ Phoromatic Server. The systemd service files will automatically be installed via the Phoronix Test Suite *install-sh* process while the Upstart jobs can be copied from *deploy/phoromatic-upstart/** to */etc/init* .
+
+#### Cache Verification
+To confirm the files accessible to Phoronix Test Suite client systems, from the Phoromatic Server web user-interface go to the *settings* page followed by the *cache settings* link to view information about the download and OpenBenchmarking.org caches. From the client systems, running **phoronix-test-suite phoromatic.explore** will also supply cache statistics.
+
+#### Log Files
+The Phoromatic Server will produce a log file of events / debugging information to *~/.phoronix-test-suite/phoromatic.log* or */var/log/phoromatic.log* depending upon the service's permissions. When running the Phoronix Test Suite Phoromatic client, the log will be written to one of the respective locations in *phoronix-test-suite.log* .
+
+#### Multi-User Accounts
+For each time a user account is made from the Phoromatic web UI's log-in page, all of the test schedules, systems, and other account information is separate to allow for a completely isolated multi-user system. If a main administrator (the one creating the account) wishes to have multiple users sharing the same account data, that user can create additional accounts from the *Users* tab of their account. The main administrator can make an additional administrator account or a "viewer" account that can consume the account's data but not create/modify the schedules, systems, or other account details.
+
+#### File Locations
+When running the Phoronix Test Suite Phoromatic Server as root, rather than using the *~/.phoronix-test-suite/* directory, the standard Linux file-system hierarchy standard is honored. The main storage path is */var/lib/phoronix-test-suite/* , the user configuration file is */etc/phoronix-test-suite.xml* , and */var/cache/phoronix-test-suite/* for cache files.
+
+#### Uploading Other Test Results
+Unscheduled test results and other results found on connected systems to a Phoromatic account can upload the data to the Phoromatic Server using the *phoronix-test-suite phoromatic.upload-result <result file identifier >* sub-command.
+
+#### User Context File Logging
+For those utilizing custom set context script files as part of the Phoromatic test schedule, any important notes / log information can be written to the file specified by the *PHOROMATIC_LOG_FILE* environment variable set while running the user context scripts. The contents of that file is then sent to the Phoromatic Server otherwise the standard output of the script's execution is submitted to the Phoromatic Server for logging. These logs can then be viewed by the Phoromatic Server along with the test results. Other environment variables accessible when running a user context script include *PHOROMATIC_TRIGGER* , *PHOROMATIC_SCHEDULE_ID* , and *PHOROMATIC_SCHEDULE_PROCESS* .
+

documentation/phoronix-test-suite-window.md

@@ -0,0 +1,40 @@
+
+# Phoronix Test Suite On Windows
+
+### Introduction
+Phoronix Test Suite 8.0 features rewritten Windows support that is at a near feature parity to the program's long-standing support for Linux, macOS, BSD and Solaris operating systems. To make it abundantly clear, if you are using a Phoronix Test Suite version pre-8.0, you are best upgrading or ideally using Phoronix Test Suite Git as the Windows support remains in very active development at the moment as of early 2018.
+
+The Phoronix Test Suite Windows support currently targets **Windows 10 x64** and **Windows Server 2016 x64** . Earlier versions of Windows, namely Windows Server 2012 and Windows 8, may work to some extent but some hardware/software reporting features and other capabilities may be missing or report warning messages. The Phoronix Test Suite Windows support is also exclusively focused on x86 64-bit support: the Phoronix Test Suite itself will run on x86 32-bit but many of the program dependencies are configured for making use of 64-bit binaries.
+
+
+### Windows Setup / Dependencies
+As with Phoronix Test Suite on Linux and other operating systems, the principal dependency is on PHP (PHP v5.3 or newer, including PHP 7.x). Running the *phoronix-test-suite.bat* file launcher for the Phoronix Test Suite on Windows will attempt to download and setup PHP on the system under *C:\PHP* as the default location should PHP support not be found within your system's *Program Files* directories. The PHP Windows build does depend upon Microsoft Visual C++ redistributable libraries, which the Windows launcher will also attempt to download and install if needed.
+
+The Phoronix Test Suite on Windows does depend upon [Cygwin](https://www.cygwin.com/) for its Bash interpreter and other basic utilities to ease the process of porting test profiles to Windows with being able to use many of the same test installation scripts on Windows/Linux/macOS/BSD/Solaris then largely unmodified. Most of the Windows tests depend upon their respective native Windows applications/binaries while this Cygwin support is a convenience for handling these Bash setup scripts and also some test profiles that depend upon a GNU toolchain. The Phoronix Test Suite will attempt to download and setup Cygwin on the system if Cygwin isn't found in its default location of *C:\cygwin64* .
+
+Various test profiles may depend upon other "external dependencies" like Python, PERL, Steam, and Java, as examples. The Phoronix Test Suite as with its support for other operating systems and Linux distributions will attempt to install these needed dependencies on a per-test basis when needed if existing support is not detected on the system.
+
+
+### Running The Phoronix Test Suite On Windows
+The Phoronix Test Suite can run from its local directory and does not need to be "installed" to a system path or any other "setup" process prior to execution. On a clean install of Windows 10 x64 or Windows Server 2016, deploying the Phoronix Test Suite is designed to be as easy and straight-forward as possible:
+
+1. Download the Phoronix Test Suite 8.0+ or [Phoronix-Test-Suite from GitHub](https://github.com/phoronix-test-suite/phoronix-test-suite) ( [zip file](https://github.com/phoronix-test-suite/phoronix-test-suite/archive/master.zip) ).
+
+2. From the Command Prompt or PowerShell, enter the *phoronix-test-suite* directory whether it be from Git or a zipped download.
+
+3. Run the *phoronix-test-suite.bat* file that should proceed to run the Phoronix Test Suite just as you would on any other operating system. If needed the Phoronix Test Suite will try to initially download and setup PHP if needed followed by the attempted automatic Cygwin setup, etc.
+
+4. Any of the Phoronix Test Suite commands from other operating systems should work on Windows. If you are new to the Phoronix Test Suite, you may enjoy a bit more guided experience by running the **phoronix-test-suite shell** command.
+
+
+### Test Profiles On Windows
+As of March 2018, around 50 of the test profiles are currently compatible with the Phoronix Test Suite on Windows. This includes many of the popular benchmarks and other interesting test cases. Over time more test profiles will continue to be ported to Windows where applicable and there are also some Windows-only tests also supported for execution by the Phoronix Test Suite.
+
+
+### Getting Started
+Besides **phoronix-test-suite shell** and **phoronix-test-suite help** , there is also **phoronix-test-suite interactive** for helping new users understand Phoronix Test Suite benchmarking. Long story short, it should be as easy as running **phoronix-test-suite benchmark c-ray** or **phoronix-test-suite benchmark crafty** as some examples for carrying out automated, cross-platform benchmarks in a side-by-side and fully-reproducible manner.
+
+
+### Support
+Community technical support is available via [GitHub](https://github.com/phoronix-test-suite/phoronix-test-suite/issues) or general inquiries via [the Phoronix Forums](https://www.phoronix.com/forums/forum/phoronix/phoronix-test-suite) . For enterprise inquiries, commercial support, and custom engineering services, [contact us](http://phoronix-test-suite.com/?k=commercial) .
+

documentation/phoronix-test-suite-windows.html

@@ -0,0 +1,27 @@
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+<Title>Phoronix Test Suite On Windows</Title>
+</head>
+<body>
+
+<h2>Introduction</h2>
+<p>Phoronix Test Suite 8.0 features rewritten Windows support that is at a near feature parity to the program's long-standing support for Linux, macOS, BSD and Solaris operating systems. To make it abundantly clear, if you are using a Phoronix Test Suite version pre-8.0, you are best upgrading or ideally using Phoronix Test Suite Git as the Windows support remains in very active development at the moment as of early 2018.</p>
+<p>The Phoronix Test Suite Windows support currently targets <strong>Windows 10 x64</strong> and <strong>Windows Server 2016 x64</strong>. Earlier versions of Windows, namely Windows Server 2012 and Windows 8, may work to some extent but some hardware/software reporting features and other capabilities may be missing or report warning messages. The Phoronix Test Suite Windows support is also exclusively focused on x86 64-bit support: the Phoronix Test Suite itself will run on x86 32-bit but many of the program dependencies are configured for making use of 64-bit binaries.</p>
+<h2>Windows Setup / Dependencies</h2>
+<p>As with Phoronix Test Suite on Linux and other operating systems, the principal dependency is on PHP (PHP v5.3 or newer, including PHP 7.x). Running the <em>phoronix-test-suite.bat</em> file launcher for the Phoronix Test Suite on Windows will attempt to download and setup PHP on the system under <em>C:\PHP</em> as the default location should PHP support not be found within your system's <em>Program Files</em> directories. The PHP Windows build does depend upon Microsoft Visual C++ redistributable libraries, which the Windows launcher will also attempt to download and install if needed.</p>
+<p>The Phoronix Test Suite on Windows does depend upon <a href="https://www.cygwin.com/">Cygwin</a> for its Bash interpreter and other basic utilities to ease the process of porting test profiles to Windows with being able to use many of the same test installation scripts on Windows/Linux/macOS/BSD/Solaris then largely unmodified. Most of the Windows tests depend upon their respective native Windows applications/binaries while this Cygwin support is a convenience for handling these Bash setup scripts and also some test profiles that depend upon a GNU toolchain. The Phoronix Test Suite will attempt to download and setup Cygwin on the system if Cygwin isn't found in its default location of <em>C:\cygwin64</em>.</p>
+<p>Various test profiles may depend upon other &quot;external dependencies&quot; like Python, PERL, Steam, and Java, as examples. The Phoronix Test Suite as with its support for other operating systems and Linux distributions will attempt to install these needed dependencies on a per-test basis when needed if existing support is not detected on the system.</p>
+<h2>Running The Phoronix Test Suite On Windows</h2>
+<p>The Phoronix Test Suite can run from its local directory and does not need to be &quot;installed&quot; to a system path or any other &quot;setup&quot; process prior to execution. On a clean install of Windows 10 x64 or Windows Server 2016, deploying the Phoronix Test Suite is designed to be as easy and straight-forward as possible:</p>
+<p>1. Download the Phoronix Test Suite 8.0+ or <a href="https://github.com/phoronix-test-suite/phoronix-test-suite">Phoronix-Test-Suite from GitHub</a> (<a href="https://github.com/phoronix-test-suite/phoronix-test-suite/archive/master.zip">zip file</a>).</p>
+<p>2. From the Command Prompt or PowerShell, enter the <em>phoronix-test-suite</em> directory whether it be from Git or a zipped download.</p>
+<p>3. Run the <em>phoronix-test-suite.bat</em> file that should proceed to run the Phoronix Test Suite just as you would on any other operating system. If needed the Phoronix Test Suite will try to initially download and setup PHP if needed followed by the attempted automatic Cygwin setup, etc.</p>
+<p>4. Any of the Phoronix Test Suite commands from other operating systems should work on Windows. If you are new to the Phoronix Test Suite, you may enjoy a bit more guided experience by running the <strong>phoronix-test-suite shell</strong> command.</p>
+<h2>Test Profiles On Windows</h2>
+<p>As of March 2018, around 50 of the test profiles are currently compatible with the Phoronix Test Suite on Windows. This includes many of the popular benchmarks and other interesting test cases. Over time more test profiles will continue to be ported to Windows where applicable and there are also some Windows-only tests also supported for execution by the Phoronix Test Suite.</p>
+<h2>Getting Started</h2>
+<p>Besides <strong>phoronix-test-suite shell</strong> and <strong>phoronix-test-suite help</strong>, there is also <strong>phoronix-test-suite interactive</strong> for helping new users understand Phoronix Test Suite benchmarking. Long story short, it should be as easy as running <strong>phoronix-test-suite benchmark c-ray</strong> or <strong>phoronix-test-suite benchmark crafty</strong> as some examples for carrying out automated, cross-platform benchmarks in a side-by-side and fully-reproducible manner.</p>
+<h2>Support</h2>
+<p>Community technical support is available via <a href="https://github.com/phoronix-test-suite/phoronix-test-suite/issues">GitHub</a> or general inquiries via <a href="https://www.phoronix.com/forums/forum/phoronix/phoronix-test-suite">the Phoronix Forums</a>. For enterprise inquiries, commercial support, and custom engineering services, <a href="http://phoronix-test-suite.com/?k=commercial">contact us</a>.</p>
+</body>
+</html>

documentation/stubs/00_overview.html

@@ -4,7 +4,7 @@
 </head>
 <body>
 <h1>Overview</h1>
-<p>The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available for Linux, Solaris, Mac OS X, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners. Version 1.0 of the Phoronix Test Suite was publicly released in 2008.</p>
+<p>The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available for Linux, Solaris, macOS, Windows, and BSD operating systems. The Phoronix Test Suite allows for carrying out tests in a fully automated manner from test installation to execution and reporting. All tests are meant to be easily reproducible, easy-to-use, and support fully automated execution. The Phoronix Test Suite is open-source under the GNU GPLv3 license and is developed by Phoronix Media in cooperation with partners. Version 1.0 of the Phoronix Test Suite was publicly released in 2008.</p>
 <p>The Phoronix Test Suite client itself is a test framework for providing seamless execution of test profiles and test suites. There are more than 200 tests available by default, which are transparently available via <a href="http://openbenchmarking.org/">OpenBenchmarking.org</a> integration. Of these default test profiles there is a range of sub-systems that can be tested and a range of hardware from mobile devices to desktops and worksrtations/servers. New tests can be easily introduced via the Phoronix Test Suite's extensible test architecture, with test profiles consisting of XML files and shell scripts. Test profiles can produce a quantitative result or other qualitative/abstract results like image quality comparisons and pass/fail. Using Phoronix Test Suite modules, other data can also be automatically collected at run-time such as the system power consumption, disk usage, and other software/hardware sensors. Test suites contain references to test profiles to execute as part of a set or can also reference other test suites. Test suites are defined via an XML schema.</p>
 <p>Running the Phoronix Test Suite for the first time can be as simple as issuing a command such as <em>phoronix-test-suite benchmark c-ray</em>, which would proceed to install a simple CPU test, execute the test, and report the results. Along with the results, the system's hardware/software information is collected in a detailed manner, relevant system logs, and other important system attributes such as compiler flags and system state. Users can optionally upload their results to OpenBenchmarking.org for sharing results with others, comparing results against other systems, and to carry out further analysis.</p>
 <h1>OpenBenchmarking.org</h1>

documentation/stubs/00_user_options.html

@@ -5,10 +5,29 @@
 </head>
 <body>
 <p>The following options are currently supported by the Phoronix Test Suite client. A list of available options can also be found by running <em>phoronix-test-suite help.</em><hr></p>
+<h1>System</h1>
+<h3>diagnostics<em>  </em>
+</h3>
+<p>This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.</p>
+<h3>interactive<em>  </em>
+</h3>
+<p>A simple text-driven interactive interface to the Phoronix Test Suite.</p>
+<h3>php-conf<em>  </em>
+</h3>
+<p>This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.</p>
+<h3>shell<em>  </em>
+</h3>
+<p>A simple text-driven shell interface / helper to the Phoronix Test Suite. Ideal for those that may be new to the Phoronix Test Suite</p>
+<h3>system-info<em>  </em>
+</h3>
+<p>Display the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.</p>
+<h3>system-sensors<em>  </em>
+</h3>
+<p>Display the installed system hardware and software sensors in real-time as detected by the Phoronix Test Suite Phodevi Library.</p>
 <h1>Test Installation</h1>
 <h3>force-install<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
-<p>This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem.</p>
+<p>This option will force the installation (or re-installation) of a test or suite. The arguments and process is similar to the install option but even if the test is installed, the entire installation process will automatically be executed. This option is generally used when debugging a test installation problem or wishing to re-install test(s) due to compiler or other environmental changes.</p>
 <h3>install<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This option will install the selected test(s) inside the testing environment directory. The install process from downloading of the test files to the installation is fully automated. The install option needs to be supplied with the test name or suite as an argument. Optionally, a OpenBenchmarking.org ID or the name of a saved results file can be supplied as well and the test(s) to install will automatically be extracted from that information. If the test is already installed and was run by the latest version of the installation process, no action will be taken. Multiple arguments can be supplied to install additional tests at the same time.</p>
@@ -46,6 +65,12 @@
 <h3>stress-run<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This option will run the passed tests/suites in the multi-process stress-testing mode. The stress-run mode will not produce a result file but is rather intended for running multiple test profiles concurrently to stress / burn-in the system. The number of tests to run concurrently can be toggled via the PTS_CONCURRENT_TEST_RUNS environment variable and by default is set to a value of 2.</p>
+<h3>strict-benchmark<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
+</h3>
+<p>This option is equivalent to the `benchmark` option except it enables various options to run benchmarks an extended number of times for ensuring better statistical accuracy if enforcing strict controls over the data quality, in some cases running the benchmarks for 20+ times.</p>
+<h3>strict-run<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
+</h3>
+<p>This option is equivalent to the `run` option except it enables various options to run benchmarks an extended number of times for ensuring better statistical accuracy if enforcing strict controls over the data quality, in some cases running the benchmarks for 20+ times.</p>
 <h1>Batch Testing</h1>
 <h3>batch-benchmark<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
@@ -65,6 +90,9 @@
 <h3>default-run<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This option will run the selected test(s). The name of the test or suite must be supplied or the OpenBenchmarking.org ID or saved local file name. Multiple arguments can be supplied to run additional tests at the same time and save the results in a suite-like fashion. Unlike the normal run option, the default-run will not prompt the user to select from the available test options but will instead use the default options as automatically set by pts-core or the test profile. Use batch-run to automatically test all of the available options.</p>
+<h3>dry-run<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
+</h3>
+<p>This option and its arguments pre-set the Phoronix Test Suite batch run mode with enforcing of defaults to not save any results and other behavior intended for a dry/test run. This option is primarily intended for testing/evaluation purposes.</p>
 <h3>internal-run<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This option and its arguments pre-set the Phoronix Test Suite batch run mode with sane values for carrying out benchmarks in a semi-automated manner and without uploading any of the result data to the public OpenBenchmarking.org.</p>
@@ -81,9 +109,6 @@
 <h3>openbenchmarking-changes<em>  </em>
 </h3>
 <p>This option will list recent changes to test profiles of enabled OpenBenchmarking.org repositories.</p>
-<h3>openbenchmarking-launcher<em>  </em>
-</h3>
-<p>This option is called automatically with the .openbenchmarking MIME file extension support for launching OpenBenchmarking.org operations.</p>
 <h3>openbenchmarking-login<em>  </em>
 </h3>
 <p>This option is used for controlling your Phoronix Test Suite client options for OpenBechmarking.org and syncing the client to your account.</p>
@@ -93,6 +118,12 @@
 <h3>openbenchmarking-repositories<em>  </em>
 </h3>
 <p>This option will list the OpenBenchmarking.org repositories currently linked to this Phoronix Test Suite client instance.</p>
+<h3>openbenchmarking-uploads<em>  </em>
+</h3>
+<p>This option will list any recent test result uploads from the system's IP address to OpenBenchmarking.org.</p>
+<h3>recently-added-tests<em>  </em>
+</h3>
+<p>This option will list the most recently added (newest) test profiles.</p>
 <h3>upload-result<em>  [Test Result]</em>
 </h3>
 <p>This option is used for uploading a test result to OpenBenchmarking.org.</p>
@@ -102,35 +133,25 @@
 <h3>upload-test-suite<em>  </em>
 </h3>
 <p>This option can be used for uploading a test suite to your account on OpenBenchmarking.org. By uploading your test suite to OpenBenchmarking.org, others are then able to browse and access this test suite for easy distribution.</p>
-<h1>System</h1>
-<h3>diagnostics<em>  </em>
-</h3>
-<p>This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.</p>
-<h3>interactive<em>  </em>
-</h3>
-<p>A simple text-driven interactive interface to the Phoronix Test Suite.</p>
-<h3>php-conf<em>  </em>
-</h3>
-<p>This option will print information that is useful to developers when debugging problems with the Phoronix Test Suite and/or test profiles and test suites.</p>
-<h3>system-info<em>  </em>
-</h3>
-<p>Display the installed system hardware and software information as detected by the Phoronix Test Suite Phodevi Library.</p>
-<h3>system-sensors<em>  </em>
-</h3>
-<p>Display the installed system hardware and software sensors in real-time as detected by the Phoronix Test Suite Phodevi Library.</p>
 <h1>Information</h1>
 <h3>info<em>  [Test | Suite | OpenBenchmarking ID | Test Result]</em>
 </h3>
 <p>This option will show details about the supplied test, suite, virtual suite, or result file.</p>
+<h3>list-all-tests<em>  </em>
+</h3>
+<p>This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories. Unlike the other test listing options, list-all-tests will show deprecated tests, potentially broken tests, or other tests not recommended for all environments. The only check in place is ensuring the test profiles are at least compatible with the operating system in use.</p>
 <h3>list-available-suites<em>  </em>
 </h3>
 <p>This option will list all test suites that are available from the enabled OpenBenchmarking.org repositories.</p>
 <h3>list-available-tests<em>  </em>
 </h3>
-<p>This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state.</p>
+<p>This option will list all test profiles that are available from the enabled OpenBenchmarking.org repositories where supported on the system and are of a verified state. If the system has no Internet access, it will only list the test profiles where the necesary test assets are available locally on the system or on an available network cache (the same behavior as using the list-cached-tests sub-command), unless using the list-all-tests option to override this behavior.</p>
 <h3>list-available-virtual-suites<em>  </em>
 </h3>
 <p>This option will list all available virtual test suites that can be dynamically created based upon the available tests from enabled OpenBenchmarking.org repositories.</p>
+<h3>list-cached-tests<em>  </em>
+</h3>
+<p>This option will list all test profiles where any needed test profiles are already cached or available from the local system under test. This is primarily useful if testing offline/behind-the-firewall and other use-cases where wanting to rely only upon local data.</p>
 <h3>list-installed-dependencies<em>  </em>
 </h3>
 <p>This option will list all of the packages / external test dependencies that are already installed on the system that the Phoronix Test Suite may potentially depend upon by test profiles.</p>
@@ -165,6 +186,9 @@
 <h3>build-suite<em>  </em>
 </h3>
 <p>This option will guide the user through the process of generating their own test suite, which they can then run. Optionally, passed as arguments can be the test(s) or suite(s) to add to the suite to be created, instead of being prompted through the process.</p>
+<h3>create-test-profile<em>  </em>
+</h3>
+<p>This option can be used for creating a Phoronix Test Suite test profile by answering questions about the test for constructing the test profile XML meta-data and handling other boiler-plate basics for getting started in developing new tests.</p>
 <h3>debug-benchmark<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This option is intended for use by test profile writers and is identical to the &lt;em&gt;run&lt;/em&gt; option but will yield more information during the run process that can be used to debug issues with a test profile or to verify the test profile is functioning correctly.</p>
@@ -180,13 +204,16 @@
 <h3>download-test-files<em>  [Test | Suite | OpenBenchmarking ID | Test Result]  ...</em>
 </h3>
 <p>This will download the selected test file(s) to the Phoronix Test Suite download cache but will not install the tests.</p>
+<h3>inspect-test-profile<em>  [Test]</em>
+</h3>
+<p>This option can be used for inspecting a Phoronix Test Suite test profile with providing inside details on test profiles for debugging / evaluation / learning purposes.</p>
 <h3>result-file-to-suite<em>  [Test Result]</em>
 </h3>
 <p>This option will guide the user through the process of generating their own test suite, which they can then run, that is based upon an existing test results file.</p>
 <h3>validate-result-file<em>  </em>
 </h3>
 <p>This option can be used for validating a Phoronix Test Suite result file as being compliant against the OpenBenchmarking.org specification.</p>
-<h3>validate-test-profile<em>  </em>
+<h3>validate-test-profile<em>  [Test]</em>
 </h3>
 <p>This option can be used for validating a Phoronix Test Suite test profile as being compliant against the OpenBenchmarking.org specification.</p>
 <h3>validate-test-suite<em>  </em>
@@ -199,6 +226,12 @@
 <h3>auto-sort-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option is used if you wish to automatically attempt to sort the results by their result identifier string.</p>
+<h3>compare-results-to-baseline<em>  [Test Result] [Test Result]</em>
+</h3>
+<p>This option will allows you to specify a result as a baseline (first parameter) and a second result file (second parameter) that will offer some analysis for showing how the second result compares to the first in matching tests.</p>
+<h3>compare-results-two-way<em>  [Test Result]</em>
+</h3>
+<p>This option will allows you to specify a result file and from there to compare two individual runs within that result file for looking at wins/losses and other metrics in a head-to-head type comparison.</p>
 <h3>edit-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option is used if you wish to edit the title and description of an existing result file.</p>
@@ -211,12 +244,15 @@
 <h3>refresh-graphs<em>  [Test Result]</em>
 </h3>
 <p>This option will re-render and save all result graphs within a saved file. This option can be used when making modifications to the graphing code or its color/option configuration file and testing the changes.</p>
-<h3>remove-from-result-file<em>  [Test Result]</em>
-</h3>
-<p>This option is used if there is a set of test results you wish to remove/delete from a saved results file. The user must specify a saved results file and then they will be prompted to select the results identifier associated with the results they wish to remove.</p>
 <h3>remove-result<em>  [Test Result]</em>
 </h3>
 <p>This option will permanently remove the saved file set that is set as the first argument.</p>
+<h3>remove-results-from-result-file<em>  [Test Result]</em>
+</h3>
+<p>This option is used if there are test results (benchmarks) to be dropped from a given result file. The user must specify a saved results file and then they will be prompted to select the tests/benchmarks to remove.</p>
+<h3>remove-run-from-result-file<em>  [Test Result]</em>
+</h3>
+<p>This option is used if there is a set of test results you wish to remove/delete from a saved results file. The user must specify a saved results file and then they will be prompted to select the results identifier associated with the results they wish to remove.</p>
 <h3>rename-identifier-in-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option is used if you wish to change the name of the identifier in a test results file that is shown in the Phoronix Test Suite Results Viewer and the contained graphs.</p>
@@ -226,6 +262,15 @@
 <h3>reorder-result-file<em>  [Test Result]</em>
 </h3>
 <p>This option is used if you wish to manually change the order in which test results are shown in the Phoronix Test Suite Results Viewer and the contained graphs. The user must specify a saved results file and then they will be prompted to select the results identifiers one at a time in the order they would like them to be displayed from left to right.</p>
+<h3>result-file-confidence<em>  [Test Result]</em>
+</h3>
+<p>This option will read a saved test results file and display various statistics on the confidence of the results with the standard deviation, three-sigma values, and other metrics while color-coding "passing" results in green.</p>
+<h3>result-file-raw-to-csv<em>  [Test Result]</em>
+</h3>
+<p>This option will read a saved test results file and output the raw result file run data to a CSV file. This raw (individual) result file output is intended for data analytic purposes where the result-file-to-csv is more end-user-ready.</p>
+<h3>result-file-stats<em>  [Test Result]</em>
+</h3>
+<p>This option is used if you wish to analyze a result file by seeing various statistics on the result data for result files containing at least two sets of data.</p>
 <h3>result-file-to-csv<em>  [Test Result]</em>
 </h3>
 <p>This option will read a saved test results file and output the system hardware and software information along with the results to a CSV output. The CSV (Comma Separated Values) output can then be loaded into a spreadsheet for easy viewing.</p>
@@ -241,10 +286,13 @@
 <h3>show-result<em>  [Test Result]</em>
 </h3>
 <p>Open up the test results in the Phoronix Test Suite Result Viewer or on OpenBenchmarking.org.</p>
-<h3>winners-and-losers<em>  [Test Result]</em>
+<h3>wins-and-losses<em>  [Test Result]</em>
 </h3>
 <p>This option is used if you wish to analyze a result file to see which runs produced the most wins/losses of those result identifiers in the saved file.</p>
 <h1>Other</h1>
+<h3>commands<em>  </em>
+</h3>
+<p>This option will display a short list of possible Phoronix Test Suite commands.</p>
 <h3>debug-dependency-handler<em>  </em>
 </h3>
 <p>This option is used for testing the distribution-specific dependency handler for external dependencies.</p>
@@ -256,14 +304,14 @@
 <p>This option is used during the development of the Phoronix Test Suite software for testing of internal interfaces, commands, and other common code-paths. The produced numbers should only be comparable for the same version of the Phoronix Test Suite, on the same hardware/software system, conducted on the same day of testing. This isn't intended as any scientific benchmark but simply to stress common PHP code-paths and looking for hot areas to optimize, etc.</p>
 <h3>help<em>  </em>
 </h3>
-<p>This option will display the list of available Phoronix Test Suite commands.</p>
+<p>This option will display a list of available Phoronix Test Suite commands and possible parameter types.</p>
 <h3>version<em>  </em>
 </h3>
 <p>This option will display the Phoronix Test Suite client version.</p>
 <h1>Modules</h1>
 <h3>auto-load-module<em>  </em>
 </h3>
-<p>This option can be used for easily adding a module to the LoadModules list in the Phoronix Test Suite user configuration file. That list controls what PTS modules are automatically loaded on start-up of the Phoronix Test Suite.</p>
+<p>This option can be used for easily adding a module to the AutoLoadModules list in the Phoronix Test Suite user configuration file. That list controls what PTS modules are automatically loaded on start-up of the Phoronix Test Suite.</p>
 <h3>list-modules<em>  </em>
 </h3>
 <p>This option will list all of the available Phoronix Test Suite modules on this system.</p>
@@ -278,7 +326,7 @@
 <p>This option can be used for debugging a Phoronix Test Suite module.</p>
 <h3>unload-module<em>  </em>
 </h3>
-<p>This option can be used for easily removing a module from the LoadModules list in the Phoronix Test Suite user configuration file. That list controls what modules are automatically loaded on start-up of the Phoronix Test Suite.</p>
+<p>This option can be used for easily removing a module from the AutoLoadModules list in the Phoronix Test Suite user configuration file. That list controls what modules are automatically loaded on start-up of the Phoronix Test Suite.</p>
 <h1>User Configuration</h1>
 <h3>enterprise-setup<em>  </em>
 </h3>
@@ -295,19 +343,13 @@
 <h3>user-config-set<em>  </em>
 </h3>
 <p>This option can be used for setting an XML value in the Phoronix Test Suite user configuration file.</p>
-<h1>Web / GUI Support</h1>
-<h3>gui<em>  </em>
-</h3>
-<p>Launch the Phoronix Test Suite HTML5 web user-interface in the local GUI mode (no remote web support) and attempt to auto-launch the web-browser. THIS FEATURE IS CURRENTLY EXPERIMENTAL AND NO LONGER ACTIVE DEVELOPMENT. See Phoronix Test Suite Phoromatic as an alternative web UI approach.</p>
-<h3>start-remote-gui-server<em>  </em>
-</h3>
-<p>Start the GUI web server and WebSocket server processes for remote (or local) access via the web-browser. The settings can be configured via the Phoronix Test Suite's XML configuration file.</p>
-<h3>start-ws-server<em>  </em>
-</h3>
-<p>Manually start a WebSocket server for communication by remote Phoronix Test Suite GUIs, the Phoronix Test Suite Multi-System Commander, and other functionality. This function checks the PTS_WEBSOCKET_PORT and PTS_WEBSOCKET_SERVER environment variables for configuration.</p>
 <h1>Phoromatic</h1>
 <h3>start-phoromatic-server<em>  </em>
 </h3>
 <p>Start the Phoromatic web server for controlling local Phoronix Test Suite client systems to facilitate automated and repeated test orchestration and other automated features targeted at the enterprise.</p>
+<h1>Result Viewer</h1>
+<h3>start-result-viewer<em>  </em>
+</h3>
+<p>Start the web-based result viewer.</p>
 </body>
 </html>

documentation/stubs/01_install.html

@@ -4,7 +4,7 @@
 </head>
 <body>
 <h1>Setup Overview</h1>
-<p>The Phoronix Test Suite supports Linux, Apple Mac OS X, Microsoft Windows, Solaris, Hurd, BSD, and other operating system environments. The only Linux distribution-specific code deals with the external dependencies support feature that are set by individual test profiles. If you are not running one of the supported Linux distributions, Solaris, BSD, or Mac OS X, you may need to install a package manually (as instructed by the Phoronix Test Suite) in order for a test to run. An example of an external dependency would be GCC and the OpenGL Utility Toolkit being needed for test profiles that build an OpenGL benchmark from source-code.</p>
+<p>The Phoronix Test Suite supports Linux, Apple macOS, Microsoft Windows, Solaris, Hurd, BSD, and other operating system environments. The only Linux distribution-specific code deals with the external dependencies support feature that are set by individual test profiles. If you are not running one of the supported Linux distributions, Solaris, BSD, or macOS, you may need to install a package manually (as instructed by the Phoronix Test Suite) in order for a test to run. An example of an external dependency would be GCC and the OpenGL Utility Toolkit being needed for test profiles that build an OpenGL benchmark from source-code.</p>
 <p>Among the distributions where the Phoronix Test Suite has been officially tested include Ubuntu, Fedora, Mandriva / Mageia, Gentoo, PCLinuxOS, Arch Linux, Pardus, OpenSuSE, Optware, webOS, Zenwalk, CentOS, Red Hat Enterprise Linux, Oracle Linux, Scientific Linux, Debian, Mint, MEPIS, Alpine Linux, Void Linux, Intel Clear Linux, and Amazon Linux EC2.</p>
 <p>Among the tested BSD distributions are FreeBSD, PC-BSD, NetBSD, OpenBSD, Debian GNU/kFreeBSD, and DragonflyBSD. Tested Solaris distributions include OpenSolaris, Solaris Express 11, Oracle Solaris 11, OpenIndiana, Illumos, and Nexenta.</p>
 <h1>Dependencies</h1>
@@ -37,10 +37,8 @@
 <p>Running <em>install-sh</em> from the root directory of the Phoronix Test Suite will install the software for system-wide access. By default the <em>phoronix-test-suite</em> executable is in <em>/usr/bin/</em>, the Phoronix Test Suite files in <em>/usr/share/phoronix-test-suite/</em>, and the documentation in /<em>usr/share/doc/phoronix-test-suite/</em>. Root access is required. The default installation prefix is /usr/ but can be adjusted as the first argument (example: <em>install-sh /home/user/</em> to install the Phoronix Test Suite in your home directory).</p>
 <h2>Debian/Ubuntu Installation</h2>
 <p>Debian/Ubuntu users are able to follow the Generic Installation instructions or can obtain a Debian Package from the Phoronix Test Suite web-site. The package contains the <em>phoronix-test-suite</em> executable in <em>/usr/bin/</em>, the Phoronix Test Suite files in <em>/usr/share/phoronix-test-suite/</em>, and the documentation in <em>/usr/share/doc/phoronix-test-suite/</em>.</p>
-<p>A phoronix-test-suite package can also be found in the official Ubuntu universe repository beginning with the Ubuntu 9.04 (Jaunty) release, but it may be out of date compared to the latest upstream version.</p>
-<p>On Ubuntu 13.04 and newer if you experience an issue of system sensor monitoring not working, other multi-process tasks, or related issues reported as PHP PCNTL missing, the pcntl functions need to be removed from the <em>disable_functions</em> line within the <em>/etc/php5/cli/php.ini</em> file. This may also affect other Linux distribution support too if the vendor disables access to the PCNTL functions.</p>
 <h2>Fedora / Red Hat Installation</h2>
-<p>The Phoronix Test Suite can be installed on Fedora, Red Hat Enterprise Linux, and CentOS systems using the generic installation method. Alternatively, a <em>phoronix-test-suite</em> package is available in recent versions of the Fedora repository and in the EPEL (Extra Packages for Enterprise Linux) repository for Red Hat Enterprise Linux. At this time, however, the Fedora-supplied package is vastly out of date.</p>
+<p>The Phoronix Test Suite can be installed on Fedora, Red Hat Enterprise Linux, and CentOS systems using the generic installation method. Alternatively, a <em>phoronix-test-suite</em> package is available in recent versions of the Fedora repository and in the EPEL (Extra Packages for Enterprise Linux) repository for Red Hat Enterprise Linux. However, at times this package may be out-of-date compared to upstream stable.</p>
 <h2>Gentoo Installation</h2>
 <p>Gentoo ebuild's for the Phoronix Test Suite have been created by users and can be currently found in the <a href="http://bugs.gentoo.org/show_bug.cgi?id=216656">Gentoo BugZilla</a>.</p>
 <h2>Mandriva Installation</h2>
@@ -50,12 +48,11 @@
 <h2>BSD Installation</h2>
 <p>The Phoronix Test Suite also supports *BSD operating systems. However, like the Solaris support, not all test profiles are compatible with BSD operating systems, but pts-core has been tested against FreeBSD 7.x/8.x/9.x, NetBSD, PC-BSD 7/8/9, DragonFlyBSD 3.0/3.2/3.4/3.6/4.2, and OpenBSD 4.4/4.6. The Phoronix Test Suite will also work with Debian GNU/kFreeBSD.</p>
 <p>For DragonFlyBSD, it's a matter of just using <em>pkg_radd</em> with php5, php5-json, php5-dom, and php5-zip or <em>pkgsrc</em>. With DragonFlyBSD using dports, the command is <em>pkg install php5 php5-dom php5-zip php5-json php5-simplexml</em>. Under FreeBSD/PC-BSD, PHP can be obtained from the <em>php5</em> and <em>php5-extensions</em> Ports collection.</p>
-<h2>Mac OS X Installation</h2>
-<p>The Phoronix Test Suite is fully supported on Apple's Mac OS X operating system as of Mac OS X 10.5 with improved support under Mac OS X 10.6. PHP ships with Mac OS X by default so it's simply a matter of downloading the Phoronix Test Suite package, extracting it, and running the executable. For tests that rely upon a compiler, Apple's XCode with GCC and LLVM can be utilized.</p>
+<h2>MacOS Installation</h2>
+<p>The Phoronix Test Suite is fully supported on Apple's macOS operating system. PHP ships with macOS by default so it's simply a matter of downloading the Phoronix Test Suite package, extracting it, and running the executable. For tests that rely upon a compiler, Apple's XCode with GCC and LLVM can be utilized.</p>
 <h2>Windows Installation</h2>
 <p>The Phoronix Test Suite support on Microsoft Windows is in development and will function just fine, but is not yet feature complete compared to the other operating systems support. Only a small subset of test profiles, mostly related to graphics, are supported. In terms of the version support, right now the Phoronix Test Suite is developed and tested solely against Microsoft Windows 7 x64 and Microsoft Windows 8 x64.</p>
 <p>At this time the Phoronix Test Suite client is dependent upon the <em>php.exe</em> executable being installed to <em>C:\Program Files (x86)\PHP\ </em> or one of its subdirectories (see PHP installation notes above). Once this requirement is met, simply download the Phoronix Test Suite package, extract it to a directory of your choice, and run <em>phoronix-test-suite.bat</em></p>
-<p>The Phoronix Test Suite also uses <a href="http://www.cpuid.com/cpuz.php">CPU-Z</a> for much of the hardware detection support on Windows and is dependent upon CPU-Z being installed to <em>C:\Program Files\CPUID\CPU-Z\cpuz.exe</em> if you wish to utilize this hardware reporting feature. </p>
 <p>NOTE: because of lack of external dependencies support, it is necessary to manually download and install the files for some test profiles, such as the Unigine graphics benchmarks. See Troubleshooting for more information.</p>
 <h2>Facebook HipHop</h2>
 <p>The Phoronix Test Suite can use Facebook's <a href="https://github.com/facebook/hiphop-php">HipHop HHVM</a> virtual machine as an alternative to the de facto PHP implementation. The Phoronix Test Suite has been tested against HHVM 2.0. If HHVM is present on the system but not PHP, it will automatically be used by the Phoronix Test Suite. Alternatively, the <em>PHP_BIN</em> environment variable can be set to point to an alternative <em>hhvm</em> binary.</p>

documentation/stubs/02_external_dependencies.html

@@ -1 +1,10 @@
-<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><title>External Dependencies</title></head><body><p>The Phoronix Test Suite has a feature known as "External Dependencies" where the Phoronix Test Suite can attempt to automatically install some of the test-specific dependencies on supported distributions. If running on a distribution where there is currently no External Dependencies profile, the needed package name(s) are listed for manual installation.</p><p>Below are a list of the operating systems that currently have external dependencies support within the Phoronix Test Suite for the automatic installation of needed test files.<hr><ul></ul><li>Alpine Linux</li><li>Amazon</li><li>Angstrom</li><li>Arch Linux</li><li>CentOS</li><li>ClearOS</li><li>ClearOS Core Server</li><li>Debian</li><li>DragonFlyBSD</li><li>Fedora</li><li>Fluxbuntu</li><li>GNU KFreeBSD</li><li>Gentoo</li><li>Goobuntu</li><li>HP</li><li>Joli Cloud</li><li>Linaro</li><li>Linux Embedded Development Environment</li><li>Linux Mint</li><li>MEPIS</li><li>Mac OS X</li><li>MacPorts</li><li>Mageia</li><li>Mandriva</li><li>MeeGo</li><li>MidnightBSD</li><li>Moblin</li><li>Mythbuntu</li><li>NetBSD</li><li>Nexenta Core</li><li>OLPC</li><li>OpenIndiana</li><li>OpenMandriva</li><li>OpenSolaris</li><li>OpenSuSE</li><li>Optware</li><li>Oracle Server</li><li>PCLinuxOS</li><li>PTS Desktop Live</li><li>PTS Linux Live</li><li>Palm</li><li>Pardus Linux</li><li>Red Hat Enterprise</li><li>Red Hat Enterprise Server</li><li>SUSE</li><li>SUSE Linux</li><li>Scientific</li><li>ScientificSL</li><li>Solus</li><li>Solus Linux</li><li>Ubuntu</li><li>Void Linux</li><li>Zenwalk</li><li>gNewSense</li></p></body></html>
+<html>
+<head>
+<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+<title>External Dependencies</title>
+</head>
+<body>
+<p>The Phoronix Test Suite has a feature known as "External Dependencies" where the Phoronix Test Suite can attempt to automatically install some of the test-specific dependencies on supported distributions. If running on a distribution where there is currently no External Dependencies profile, the needed package name(s) are listed for manual installation.</p>
+<p>Below are a list of the operating systems that currently have external dependencies support within the Phoronix Test Suite for the automatic installation of needed test files.<hr><ul></ul><li>Alpine Linux</li><li>Amazon</li><li>Angstrom</li><li>Arch Linux</li><li>CentOS</li><li>ClearOS</li><li>ClearOS Core Server</li><li>Debian</li><li>DragonFlyBSD</li><li>Fedora</li><li>Fluxbuntu</li><li>GNU KFreeBSD</li><li>Gentoo</li><li>Goobuntu</li><li>HP</li><li>Joli Cloud</li><li>Linaro</li><li>Linux Embedded Development Environment</li><li>Linux Mint</li><li>MEPIS</li><li>Mac OS X</li><li>MacPorts</li><li>Mageia</li><li>Mandriva</li><li>MeeGo</li><li>Microsoft Windows</li><li>MidnightBSD</li><li>Moblin</li><li>Mythbuntu</li><li>NetBSD</li><li>Nexenta Core</li><li>OLPC</li><li>OpenIndiana</li><li>OpenMandriva</li><li>OpenMandrivaLinux</li><li>OpenSolaris</li><li>OpenSuSE</li><li>Optware</li><li>Oracle Server</li><li>PCLinuxOS</li><li>PTS Desktop Live</li><li>PTS Linux Live</li><li>Palm</li><li>Pardus Linux</li><li>Red Hat Enterprise</li><li>Red Hat Enterprise Server</li><li>SUSE</li><li>SUSE Linux</li><li>Scientific</li><li>ScientificSL</li><li>Solus</li><li>Solus Linux</li><li>Termux</li><li>Ubuntu</li><li>Void Linux</li><li>Zenwalk</li><li>gNewSense</li><li>macOS Brew</li></p>
+</body>
+</html>

documentation/stubs/40_configuration.html

@@ -55,8 +55,14 @@
 <p>When running any test(s), if you would like the test(s) to continue running for a number of times, this variable can be used. The value should be the number of times to loop the testing process before ending.</p>
 <p><strong>FORCE_TIMES_TO_RUN</strong></p>
 <p>If you wish to override the number of times to run each test -- rather than the Phoronix Test Suite using the number of times specified in each test profile -- this variable can be used.</p>
+<p><strong>FORCE_TIMES_TO_RUN_MULTIPLE</strong></p>
+<p>This option allows specifying a multiple for increasing the number of times a test will run based upon the original TimesToRun value specified in the test definition. This allows for increasing the expected times to run based on a multiple of that default rather than a static value.</p>
 <p><strong>FORCE_MIN_TIMES_TO_RUN</strong></p>
 <p>This is similar to the FORCE_TIMES_TO_RUN option but will only be used if the test profile's run count is less than this defined value.</p>
+<p><strong>FORCE_MIN_TIMES_TO_RUN_CUTOFF</strong></p>
+<p>When used in conjunction with FORCE_MIN_TIMES_TO_RUN, the override value will only be applied to test profiles where its average run-time length (in minutes) is less than the value specified by FORCE_MIN_TIMES_TO_RUN_CUTOFF.</p>
+<p><strong>IGNORE_RUNS</strong></p>
+<p>IGNORE_RUNS can be passed a comma-separated list of runs to skip on each benchmark. For example, IGNORE_RUNS=1 would always drop the first run from being recorded.</p>
 <p><strong>NO_FILE_HASH_CHECKS</strong></p>
 <p>To disable MD5/SHA256 check-sums from being checked when downloading test files, set this variable to 1. This variable used to be known as <em>NO_MD5_CHECKS</em>, which is still honored but was changed to <em>NO_FILE_HASH_CHECKS</em> to reflect other kind of file hash sum checks.</p>
 <p><strong>NO_HTTPS</strong></p>

documentation/stubs/68_offline_additions.html

@@ -0,0 +1,18 @@
+<html>
+<head>
+<Title>Offline Improvements + Confidential Testing / Avoiding Accidental Result Uploads</Title>
+</head>
+<body>
+<h1>Offline Enhancement Via Local Cache</h1>
+<p>Beginning with Phoronix Test Suite 9.0, there are improvements to improve the out-of-the-box experience if running the Phoronix Test Suite in a strictly offline environment / behind-the-firewall without access to OpenBenchmarking.org for being able to obtain test profiles / test suites. From Phoronix Test Suite 3.0 when OpenBenchmarking.org was introduced until Phoronix Test Suite 9.0, Internet connectivity was initially required for obtaining the test profiles/suites as the cloud/repository. OpenBenchmarking.org allows for tests to be updated independently of the Phoronix Test Suite releases as well as allowing new tests to be introduced on-demand. Aside from when new tests require explicit new PTS features, this allows tests/suites to be seamlessly used by older versions of the Phoronix Test Suite without any upgrade process required, assuming Internet connectivity is available.</p>
+<p>Beginning with Phoronix Test Suite 9.0, a static snapshot of the official tests/suites is included as part of the Phoronix Test Suite package. The intention with this is to provide a static snapshot with all tests/suites as of release time, similar to the behavior with pre-3.0 releases. The benefit to including this static snapshot is helping those that are running strictly offline/isolated to be able to have at least recent tests/suites available without first needing to query OpenBenchmarking.org for this data. But Internet support is certainly desired in order to be able to obtain updated and new test profiles.</p>
+<p>This static snapshot is provided in the <em>ob-cache/</em> Phoronix Test Suite folder. If this cache is not needed or wish to customize/extend it, it can be safely removed and or altered without causing issues. When the Phoronix Test Suite has Internet connectivity, it will continue to query OpenBenchmarking.org for new/updated tests and suites.</p>
+<p>This local cache does provide current and previous versions of test profiles to allow users to continue running older versions of tests/results even when upgrading their Phoronix Test Suite offline copy.</p>
+<p>Even with the local cache, there still is the need for obtaining any necessary files needed to run the selected test(s). For those wishing to optimize that workflow for offline usage, see the existing <em>phoronix-test-suite make-download-cache</em> sub-command documentation. The <em>phoronix-test-suite make-openbenchmarking-cache</em> sub-command may also be desirable depending upon setup.</p>
+<h1>Disabling OpenBenchmarking.org Result Upload Functionality</h1>
+<p>Phoronix Test Suite 9.0 also improved the workflow around disabling OpenBenchmarking.org result uploading functionality for those carrying out confidential tests or otherwise wish to provide safeguards for ensuring no results may be accidentally uploaded publicly.</p>
+<p>Removal of OpenBenchmarking.org upload support can be done by deleting <em>phoronix-test-suite/pts-core/objects/pts_openbenchmarking_upload.php</em>. If that file is removed, the Phoronix Test Suite should respond gracefully and not prompt users about any upload and within that file is the only logic for actually uploading the results to Openbenchmarking. So simply by removing that file you should be covered from any accidental uploading of results. Removal/disabling of this file also prevents any anonymous usage reporting.</p>
+<p>For those without the ability to remove that file from their Phoronix Test Suite installation or as a secondary safeguard, from the Phoronix Test Suite user configuration file (<em>/etc/phoronix-test-suite.xml<em> as root or <em>~/.phoronix-test-suite/user-config.xml<em> for most users) is a "<em>AllowResultUploadsToOpenBenchmarking</em>" option. If setting that value to <em>FALSE</em>, it should apply the same behavior as if deleting the pts_openbenchmarking_upload file.</p>
+<p>If distributing a customized/local copy of the Phoronix Test Suite, the default behavior of the configuration file (in addition to deleting the pts_openbenchmarking_upload file) can be done via the user configuration defaults defined within <em>pts-core/static/user-config-defaults.xml</em>.</p>
+</body>
+</html>

documentation/stubs/69_creating_tests.html

@@ -0,0 +1,72 @@
+<html>
+<head>
+<Title>Creating Test Profiles</Title>
+</head>
+<body>
+<h1>Creating A New Phoronix Test Suite / OpenBenchmarking.org Test Profile</h1>
+<p>A test profile itself consists of a main XML file containing the test's meta-data (<em>test-definition.xml</em>), an XML file if needed to specify required files, download URLs and their MD5/SHA256 hashes and file sizes (<em>downloads.xml</em>), and the <em>install.sh</em> script for installing the test and generating the run-script for execution by the Phoronix Test Suite at run-time. There are also other potential files like <em>pre.sh</em>, <em>interim.sh</em>, and <em>post.sh</em> for execution at pre-run, interim-run, and post-run stages by the Phoronix Test Suite. With the script files comprising test profiles, there is also the option of post-fixing them with e.g. <em>_linux</em> or <em>_windows</em> if wishing to supply different script files based upon the operating system being run during testing, if the setup steps may differ greatly based upon MacOS/Windows/Linux/BSD/Solaris platforms.</p>
+<p>Traditionally the easiest way to learn/understand test profile development has been by looking at looking at a basic test like <em>c-ray</em>, <em>tesseract</em> or the hundreds of others that are publicly available. Simply run for example <em>phoronix-test-suite benchmark tesseract</em> and then look at the test profile's contents either via the default locations in <em>~/.phoronix-test-suite/test-profiles</em> or <em>/var/lib/phoronix-test-suite/test-profiles/</em> if running as root on non-Windows platforms.</p>
+<h2>Versioning Mandate</h2>
+<p>The Phoronix Test Suite / OpenBenchmarking.org philosophy <em>mandates that with any change, a new version of the test profile be tagged</em>. This is done for reproducibility and being able to ensure the exact test profile state when a set of tests are conducted. Test profiles are versioned in a <strong>test-X.Y.Z</strong> format where <em>X</em> or <em>Y</em> are bumped whenever a change is made to the test profile that makes the results incomparable to a previous version of the tests (e.g. updating against a new upstream code-base, adjusting the parameters to what is benchmarked, etc). The Phoronix Test Suite then knows not to attempt any comparisons with an incompatible version difference or to go and fetch that specific version of the package. The <em>Z</em> is bumped when just making non-important changes such as just updating the test's meta-data, changing download URLs, etc. This is easy to enforce with the OpenBenchmarking.org infrastructure rather than having to worry about non-human-friendly Git hashes as test profile versions or having to create Git tags after every commit. The test profile version can optionally be specified when running a test, e.g. <em>phoronix-test-suite benchmark scimark2-1.2.1</em> instead of <em>phoronix-test-suite benchmark scimark2</em>, which would by default choose the latest available test profile version from OpenBenchmarking.org or the latest version on any local Phoromatic Server. The test profile versions are also always written out as part of the Phoronix Test Suite result XML data.</p>
+<h2>Writing A Sample Program</h2>
+<p>Writing a test profile for the Phoronix Test Suite is a relatively quick and easy process for anyone familiar with common Linux commands and the basics of XML. To help you understand the design of the Phoronix Test Suite, this guide covers the steps needed to write a testing profile for a very simple application.</p>
+<p>The first step in the profile writing process is to, well, have a piece of software you'd like to use with the Phoronix Test Suite. This software can be closed-source or open-source and be virtually anything as long as it is compatible with an operating system that is supported by the Phoronix Test Suite.</p>
+<p>For this guide, the piece of software being used for demonstration is just a simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz formula. Below is this sample piece of software intended just for demonstration purposes.</p>
+<blockquote>#include &lt;iostream&gt;<br>
+#include &lt;math.h&gt;</p>
+<p>int main()<br>
+{<br>
+double pi = 0;</p>
+<p> for(long int i = 1; i &lt;= 87654321; i++)<br>
+pi += (double) pow(-1, i + 1) / (2 * i - 1);</p>
+<p> pi *= 4;<br>
+std::cout &lt;&lt; &quot;Done Calculating Pi...&quot; &lt;&lt; endl;<br>
+return 0;<br>
+}</blockquote>
+<p>The first step in the actual profile writing process is to name it. If you're  looking to ultimately push this profile to be included in the Phoronix Test Suite, its name must be all lower case and consist of just alpha-numeric characters, but can contain dashes (-). A more advanced test profile capability is operating system prefixes, and if using those there is an underscore separating the prefix from the normal profile name. For this sample profile, we're calling it <em>sample-program</em> and the file-name would be <em>sample-program/test-definition.xml</em>. Our (very basic) profile is showcased below.</p>
+<blockquote>&lt;PhoronixTestSuite&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestProfile&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Version&gt;1.1.0&lt;/Version&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestType&gt;Processor&lt;/TestType&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;SoftwareType&gt;Utility&lt;/SoftwareType&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;License&gt;FREE&lt;/License&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Status&gt;PRIVATE&lt;/Status&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Maintainer&gt;Phoronix Media&lt;/Maintainer&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/TestProfile&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TestInformation&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Title&gt;Sample Pi Program&lt;/Title&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;TimesToRun&gt;3&lt;/TimesToRun&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ResultScale&gt;Seconds&lt;/ResultScale&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Proportion&gt;LIB&lt;/Proportion&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Description&gt;A simple C++ program that calculates Pi to 8,765,4321 digits using the Leibniz formula. This test can be used for showcasing how to write a basic test profile.&lt;/Description&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ExternalDependencies&gt;build-utilities&lt;/ExternalDependencies&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/TestInformation&gt;<br>
+&lt;/PhoronixTestSuite&gt;</blockquote>
+<p>This XML profile is what interfaces with the Phoronix Test Suite and provides all the needed information about the test as well as other attributes. For a complete listing of all the supported profile options, look at the specification files in the documentation folder. In the case of <em>sample-program</em>, it lets the Phoronix Test Suite know that it's composed of free software, is designed to test the processor, is intended for private use only, and this profile is maintained by Phoronix Media. In addition, it tells the Phoronix Test Suite to execute this program three times and as no result quantifier is set, the average of the three runs will be taken. This profile also tells the Phoronix Test Suite that the generic <em>build-utilities</em> package is needed, which will attempt to ensure that default system C/C++ compiler and the standard development utilities/libraries are installed on your Linux distribution. This is needed as the C++ source-code will need to be built from source.</p>
+<p>The next step is to write the <em>install.sh</em> file, which once called by the Phoronix Test Suite is intended to install the test locally for benchmarking purposes. The <em>install.sh</em> file is technically optional, but is generally used by all tests. Note: The first argument supplied to the install script is the directory that the test needs to be installed to. The <em>install.sh</em> file (in our instance) is to be placed inside <em>test-profiles/sample-program</em>. Below is the <em>install.sh</em> for the <em>sample-program</em>.</p>
+<blockquote>#!/bin/sh<br /><br />
+tar -xjf sample-pi-program-1.tar.bz2<br />
+g++ sample-pi-program.cpp -o sample-pi-program<br />
+echo &quot;#!/bin/sh<br />
+./sample-pi-program 2&gt;&amp;1<br />
+&quot; &gt; sample-program<br>
+chmod +x sample-program</blockquote>
+<p>This install file builds the code with GCC, and then creates a small script that is run by the Phoronix Test Suite. Where does the source-code come into play? Well, it needs to be downloaded now from a web server. The Phoronix Test Suite has built-in support for managing downloads from multiple servers in a random over, fall-back support if one mirror is done, and verification of MD5 check-sums. Below is the <em>downloads.xml</em> file for <em>sample-program</em> that covers all of this.</p>
+<blockquote>&lt;PhoronixTestSuite&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Downloads&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;Package&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;URL&gt;http://www.phoronix-test-suite.com/benchmark-files/sample-pi-program.cpp&lt;/URL&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;MD5&gt;e90fb790df8d1544696a1439c9b5bd8d&lt;/MD5&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/Package&gt;<br>
+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/Downloads&gt;<br>
+&lt;/PhoronixTestSuite&gt;</blockquote>
+<p>The final step in the profile writing process is to write a parser to strip all information but the reported result from the standard output or <em>$LOG_FILE</em>. In the case of a test profile just measuring how long it takes to run, it is as simple as a <em>results-definition.xml</em> looking like:</p>
+<blockquote>&lt;?xml version="1.0"?&gt;<br>
+&lt;PhoronixTestSuite&gt;<br>
+  &lt;SystemMonitor&gt;<br>
+    &lt;Sensor&gt;sys.time&lt;/Sensor&gt;<br>
+  &lt;/SystemMonitor&gt;<br>
+&lt;/PhoronixTestSuite&gt;</blockquote>
+<p>After that, with all the files in their correct locations, just run: <em>phoronix-test-suite benchmark sample-program</em>. The Phoronix Test Suite should now handle the rest by installing the test, running the test, and recording the results (if you so choose). There is no additional work that needs to be done for the results to be recorded in the results viewer or even reporting the results to OpenBenchmarking.org. An up-to-date version of this test profile can be run via <em>phoronix-test-suite benchmark sample-program</em> and then by looking at the test profile source via <em>~/.phoronix-test-suite/test-profiles/pts/sample-program*</em> or within <em>/var/lib/phoronix-test-suite/test-profiles/pts/</em> if running as root.</p>
+</body>
+</html>

documentation/stubs/69_pts_os.html

@@ -1,20 +0,0 @@
-<html>
-<head>
-<Title>PTS Desktop Live</Title>
-</head>
-<body>
-<h1>PTS Desktop Live 2010.1 &quot;Anzhofen&quot;</h1>
-<p>PTS Desktop Live and PTS Netbook Live are Linux-based operating systems designed to provide a free software platform for carrying out system tests and hardware benchmarks in an automated, reproducible, and effective manner. The Phoronix Test Suite operating systems are designed around the <a href="http://www.phoronix-test-suite.com/">Phoronix Test Suite</a> with the operating system itself consisting of a stripped down Linux environment, then built upon that is the Phoronix Test Suite with its test profiles and suites. This is not a standard Linux desktop with the normal throng of packages.</p>
-<p>Testing with PTS Desktop Live is intended to be via a Live DVD or Live USB (i.e. flash drive or memory card) environment. PTS Netbook Live is similar to PTS Desktop Live but designed for testing on lightweight x86 systems and netbooks. Both PTS Desktop Live and PTS Netbook Live make it very easy to deploy the Phoronix Test Suite across an array of systems, isolates any software differences when seeking to perform hardware comparisons, and is designed to be very easy to carry out these Linux benchmarks even for a first-time Linux user.</p>
-<p>The inaugural release of this operating system was PTS Desktop Live 2009.3 (codenamed &quot;Gernlinden&quot;) and was based upon the Phoronix Test Suite 2.0 release. The Gernlinden release shipped with more than 50 open-source test profiles and all needed files being located on the disc image. PTS Desktop Live uses the GTK2 GUI dialog for the Phoronix Test Suite by default, but the CLI options are also accessible. PTS Desktop Live also features optimizations to the Linux kernel and related components for use on the latest high-end hardware and other performance related tweaking. PTS Netbook Live features optimizations for the Intel Atom processor.</p>
-<p>The mission of the Phoronix Test Suite and related products will not be achieved until it has made Linux benchmarking incredibly robust and innovative, drop dead easy to perform nearly any kind of benchmark, and when it out-paces other leading operating systems for its benchmarking abilities. These operating systems are being updated quarterly in conjunction with new Phoronix Test Suite releases. The Phoronix Test Suite will continue to be supported under all other Linux distributions, OpenSolaris, Mac OS X, Windows, and *BSD operating systems, but this is just another addition on top of that.</p>
-<p>PTS Desktop Live is designed to run on x86_64 compatible systems with 2GB or more of system memory (ideally 4GB+ if you wish to run many of the tests in a live environment) being highly recommended along with a modern CPU and graphics processor. No proprietary graphics drivers are shipped with PTS Desktop Live at this time, so depending upon the hardware you may not have OpenGL acceleration to run the few graphics-related tests that are included with this release. Only graphics drivers are provided for ATI/AMD, NVIDIA, and Intel hardware. An Internet connection is not required but is needed for communication with OpenBenchmarking.org and for utilizing some of the other Phoronix Test Suite features.</p>
-<p>PTS Netbook Live is designed for netbooks and nettop computers with an Intel Atom processor.</p>
-<h2>Quick Start Guide</h2>
-<p>When the operating system has fully booted, the end-user agreement for the Phoronix Test Suite will be displayed, followed by initializing the main GTK user interface for the Phoronix Test Suite. The Phoronix Test Suite has more than 100 test profiles while there are about 50 test profiles featured within live environment, which are free software, do not pose significant install size problems, and are suitable for benchmarking from a live environment. By default, PTS Desktop Live and PTS Netbook Live will only show the test profiles and suites where all of the needed test files and dependencies ship with this open-source operating system. However, to access any of the test profiles that may require downloading additional files or external dependencies, use the <em>View</em> menu to toggle the different options. The system's hardware and software information can also be viewed in a graphical window from this menu.</p>
-<p>From the <em>Available Tests</em> tab, any test/suite can be selected to view more information. If you wish to install the test, simply click on the install button while waiting shortly for it to install. Afterwards, the test can be run by selecting it again and clicking on the run/benchmark button. If you have a <a href="http://www.openbenchmarking.org/">OpenBenchmarking.org</a> account, you can login from the <em>File</em> menu while Phoronix Test Suite settings are adjustable from the <em>Edit</em> menu.</p>
-<p>When the testing has completed, the graphical user interface will reappear. Via the buttons that appear at the top of the interface, the test results can then be opened within the web browser. Depending upon the test profile, it may also be possible to immediately compare the test result(s) against a set of reference systems. If the completed test(s) do have reference results available, a list of the comparable systems are shown, so that you can click on one or more of these reference systems to see how the given system performs in comparison. Results from OpenBenchmarking.org can also be compared with any of the test results.</p>
-<p>Assuming you did not launch any commands from the terminal to alter the disk(s) or its contents or carry out any other changes; PTS Desktop Live will have not touched your data or the installed operating system(s). If you wish to archive any of your test results, upload the results to OpenBenchmarking.org. Alternatively, when going to the <em>Test Results</em> tab and selecting a test, the <em>File</em> menu presents options for exporting results to text, CSV, and PDF files. The contents of the <em>~/.phoronix-test-suite/test-results/</em> folder can also be backed up for preserving the test results.</p>
-<p>If you wish to use the Phoronix Test Suite from the command-line, open a terminal and run <em>phoronix-test-suite</em>.</p>
-</body>
-</html>

documentation/stubs/readme-basics.txt

@@ -2,17 +2,17 @@ The Phoronix Test Suite itself is an open-source framework for conducting automa
 
 This framework is designed to be an extensible architecture so that new test profiles and suites can be easily added to represent performance benchmarks, unit tests, and other quantitative and qualitative (e.g. image quality comparison) measurements. Available through OpenBenchmarking.org, a collaborative storage platform developed in conjunction with the Phoronix Test Suite, are more than 200 individual test profiles and more than 60 test suites available by default from the Phoronix Test Suite. Independent users are also able to upload their test results, profiles, and suites to OpenBenchmarking.org. A test profile is a single test that can be executed by the Phoronix Test Suite -- with a series of options possible within every test -- and a test suite is a seamless collection of test profiles and/or additional test suites. A test profile consists of a set of Bash/shell scripts and XML files while a test suite is a single XML file.
 
-[OpenBenchmarking.org](http://www.openbenchmarking.org/) also allows for conducting side-by-side result comparisons, a central location for storing and sharing test results, and collaborating over test data. [Phoromatic](http://www.phoromatic.com/) is a complementary platform to OpenBenchmarking.org and the Phoronix Test Suite for interfacing with Phoronix Test Suite client(s) to automatically execute test runs on a timed, per-commit, or other trigger-driven basis. Phoromatic is designed for enterprise and allows for the easy management of multiple networked systems running Phoronix Test Suite clients via a single web-based interface.
+[OpenBenchmarking.org](https://www.openbenchmarking.org/) also allows for conducting side-by-side result comparisons, a central location for storing and sharing test results, and collaborating over test data. [Phoromatic](https://www.phoromatic.com/) is a complementary platform to OpenBenchmarking.org and the Phoronix Test Suite for interfacing with Phoronix Test Suite client(s) to automatically execute test runs on a timed, per-commit, or other trigger-driven basis. Phoromatic is designed for enterprise and allows for the easy management of multiple networked systems running Phoronix Test Suite clients via a single web-based interface.
 
-Professional support and custom engineering for the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org is available by contacting <http://www.phoronix-test-suite.com/>.
+Professional support and custom engineering for the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org is available by contacting <https://www.phoronix-test-suite.com/>.
 
 Full details on the Phoronix Test Suite setup and usage is available from the included HTML/PDF documentation within the phoronix-test-suite package and from the Phoronix Test Suite website.
 
 ## Installation & Setup
 
-The Phoronix Test Suite is supported on Linux, *BSD, Solaris, Mac OS X, and Windows systems. However, the most full-featured and well supported operating system for conducting the tests is Linux with some non-basic functionality not being available under all platforms. The Phoronix Test Suite software/framework is compatible with all major CPU architectures (e.g. i686, x86_64, ARM, PowerPC), but not all of the test profiles/suites are compatible with all architectures.
+The Phoronix Test Suite is supported on Linux, *BSD, Solaris, macOS, and Windows systems. However, the most full-featured and well supported operating system for conducting the tests is Linux with some non-basic functionality not being available under all platforms. The Phoronix Test Suite software/framework is compatible with all major CPU architectures (e.g. i686, x86_64, ARM, RISC-V, PowerPC), but not all of the test profiles/suites are compatible with all architectures.
 
-The Phoronix Test Suite can be installed for system-wide usage or run locally without installation from the extracted tar.gz/zip package. The only hard dependency on the Phoronix Test Suite is having command-line support for PHP (PHP 5.3+) installed. A complete PHP stack (e.g. with web server) is **not** needed, but merely the PHP command-line support, which is widely available from operating system package managers under the name `php`, `php5-cli`, or `php5`.
+The Phoronix Test Suite can be installed for system-wide usage or run locally without installation from the extracted tar.gz/zip package. The only hard dependency on the Phoronix Test Suite is having command-line support for PHP (PHP 5.3+) installed. A complete PHP stack (e.g. with web server) is **not** needed, but merely the PHP command-line support, which is widely available from operating system package managers under the name `php`, `php7-cli`, `php5-cli`, or `php7`.
 
 ## Usage
 
@@ -20,6 +20,6 @@ The process to download, install/setup, execute, and report the results of a ben
 
 Viewing installed system hardware and software is available via `phoronix-test-suite system-info`.
 
-Facilitating a result comparison from OpenBenchmarking.org can be done by running, for example, `phoronix-test-suite benchmark 1204293-BY-PHORONIX357` if wishing to compare the results of the `http://openbenchmarking.org/result/1204293-BY-PHORONIX357` result file.
+Facilitating a result comparison from OpenBenchmarking.org can be done by running, for example, `phoronix-test-suite benchmark 1204293-BY-PHORONIX357` if wishing to compare the results of the `https://openbenchmarking.org/result/1204293-BY-PHORONIX357` result file.
 
-Additional information is available from the Phoronix Test Suite website <http://www.phoronix-test-suite.com/> and the material bundled within the `phoronix-test-suite/documentation/` directory or via <http://www.phoronix-test-suite.com/documentation/>. A man page is also bundled with the phoronix-test-suite software.
+Additional information is available from the Phoronix Test Suite website <https://www.phoronix-test-suite.com/> and the material bundled within the `phoronix-test-suite/documentation/` directory or via <https://www.phoronix-test-suite.com/documentation/>. A man page is also bundled with the phoronix-test-suite software.

install-sh

@@ -2,7 +2,7 @@
 
 # Phoronix Test Suite
 # URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
-# Copyright (C) 2008 - 2018, Phoronix Media
+# Copyright (C) 2008 - 2019, Phoronix Media
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -84,6 +84,7 @@ cp deploy/*-systemd/*.service $DESTDIR$INSTALL_PREFIX/../usr/lib/systemd/system/
 
 rm -rf $DESTDIR$INSTALL_PREFIX/share/phoronix-test-suite/pts-core
 cp -r pts-core $DESTDIR$INSTALL_PREFIX/share/phoronix-test-suite/
+cp -r ob-cache $DESTDIR$INSTALL_PREFIX/share/phoronix-test-suite/
 cp -r deploy $DESTDIR$INSTALL_PREFIX/share/phoronix-test-suite/
 rm -f $DESTDIR$INSTALL_PREFIX/share/phoronix-test-suite/pts-core/static/phoronix-test-suite.desktop
 rm -f $DESTDIR$INSTALL_PREFIX/share/phoronix-test-suite/pts-core/static/phoronix-test-suite-launcher.desktop

install.bat

@@ -0,0 +1,33 @@
+@echo off
+
+:: Phoronix Test Suite
+:: URLs: http://www.phoronix.com, http://www.phoronix-test-suite.com/
+:: Copyright (C) 2018, Phoronix Media
+::
+:: This program is free software; you can redistribute it and/or modify
+:: it under the terms of the GNU General Public License as published by
+:: the Free Software Foundation; either version 3 of the License, or
+:: (at your option) any later version.
+::
+:: This program is distributed in the hope that it will be useful,
+:: but WITHOUT ANY WARRANTY; without even the implied warranty of
+:: MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+:: GNU General Public License for more details.
+::
+:: You should have received a copy of the GNU General Public License
+:: along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+:: Generic Phoronix Test Suite installer for Windows
+
+:: Ensure the user is in the correct directory
+If Not Exist "pts-core\phoronix-test-suite.php" (
+echo "To install the Phoronix Test Suite you must first change directories to phoronix-test-suite. For support visit: http://www.phoronix-test-suite.com/"
+exit
+)
+set destination="C:\phoronix-test-suite"
+md %destination%
+:: cd /d %destination%
+:: for /F "delims=" %%i in ('dir /b') do (rmdir "%%i" /s/q || del "%%i" /s/q)
+
+xcopy %cd% %destination% /E
+echo Phoronix Test Suite installed to %destination%

ob-cache/NOTICE.txt

@@ -0,0 +1,5 @@
+The files in this directory are optional, providing a static cache of test profiles / test suites available via OpenBenchmarking.org.
+
+Patches to this directory will not be accepted but test profile changes should be submitted via https://github.com/phoronix-test-suite/test-profiles/
+
+This "ob-cache" is intended to help those running the Phoronix Test Suite behind firewalls / offline where they may not be able to query OpenBenchmarking.org for obtaining test data.

ob-cache/openbenchmarking.org/git.index

@@ -0,0 +1 @@
+{"main":{"generated":1567189513},"tests":{"aom-av1":{"title":"AOM AV1","description":"This is a simple test of the AOMedia AV1 encoder run on the CPU with a sample video file.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux","MacOSX","BSD"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"480","last_updated":"1552063091","first_added":"1552063091","popularity":"874","downloads":"1178","versions":["1.1.0"]},"svt-av1":{"title":"SVT-AV1","description":"This is a test of the Intel Open Visual Cloud Scalable Video Technology SVT-AV1 CPU-based multi-threaded video encoder for the AV1 video format with a sample 1080p YUV video file. This test profile fork builds the encoder from Git source rather than a snapshot.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"114","last_updated":"1552059151","first_added":"1552059151","popularity":"866","downloads":"1176","versions":["1.2.0"]},"svt-hevc":{"title":"SVT-HEVC","description":"This is a test of the Intel Open Visual Cloud Scalable Video Technology SVT-HEVC CPU-based multi-threaded video encoder for the HEVC \/ H.265 video format with a sample 1080p YUV video file. This test uses SVT-HEVC from Git master.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"55","last_updated":"1552060068","first_added":"1552060068","popularity":"872","downloads":"1171","versions":["1.0.0"]},"svt-vp9":{"title":"SVT-VP9","description":"This is a test of the Intel Open Visual Cloud Scalable Video Technology SVT-VP9 CPU-based multi-threaded video encoder for the VP9 video format with a sample 1080p YUV video file. This test profile uses the Git snapshot of SVT-VP9.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"65","last_updated":"1552059681","first_added":"1552059681","popularity":"873","downloads":"1165","versions":["1.0.0"]},"vpxenc":{"title":"VP9 libvpx Encoding","description":"This is a standard video encoding performance test of Google's libvpx library and the vpxenc command for the VP9\/WebM format using a sample 1080p video.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"744","last_updated":"1552061773","first_added":"1552061773","popularity":"883","downloads":"1151","versions":["2.3.0"]},"x265":{"title":"x265","description":"This is a simple test of the x265 encoder run on the CPU with a sample 1080p video file.","test_type":"Processor","software_type":"Utility","status":"Verified","supported_platforms":["Linux","MacOSX","BSD"],"internal_tags":["SMP"],"test_extends":"","package_hash":null,"average_run_time":"93","last_updated":"1552061034","first_added":"1552061034","popularity":"864","downloads":"1209","versions":["1.1.0"]}},"suites":[]}

ob-cache/test-profiles/git/aom-av1-1.1.0/downloads.xml

@@ -0,0 +1,13 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://ultravideo.cs.tut.fi/video/Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z</URL>
+      <MD5>84ae521c95aa2537e16b34bbf72f2def</MD5>
+      <SHA256>e2f60b904789a60f6d1edc194d8540d401dd882e3ee3605b9b1de8feacc72133</SHA256>
+      <FileName>Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z</FileName>
+      <FileSize>676792531</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/aom-av1-1.1.0/install.sh

@@ -0,0 +1,18 @@
+#!/bin/sh
+
+rm -rf aom-master
+git clone https://aomedia.googlesource.com/aom aom-master
+mkdir aom-master/build
+cd aom-master/build
+cmake ..
+make -j $NUM_CPU_CORES
+echo $? > ~/install-exit-status
+cd ~
+
+7z x Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z
+
+echo "#!/bin/sh
+./aom-master/build/aomenc -v --rt --threads=\$NUM_CPU_CORES --tile-columns=2 --limit=20 -o test.av1 Bosphorus_1920x1080_120fps_420_8bit_YUV.y4m > 1.log 2>&1
+echo \$? > ~/test-exit-status
+sed \$'s/[^[:print:]\t]/\\n/g' 1.log > \$LOG_FILE" > aom-av1
+chmod +x aom-av1

ob-cache/test-profiles/git/aom-av1-1.1.0/results-definition.xml

@@ -0,0 +1,9 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Pass 1/2 frame    3/4        768B    2048b/f   61440b/s  165907 us #_RESULT_# fps)</OutputTemplate>
+    <LineHint>fps</LineHint>
+    <TurnCharsToSpace>(</TurnCharsToSpace>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/aom-av1-1.1.0/test-definition.xml

@@ -0,0 +1,26 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>AOM AV1</Title>
+    <AppVersion>Git</AppVersion>
+    <Description>This is a simple test of the AOMedia AV1 encoder run on the CPU with a sample video file.</Description>
+    <ResultScale>Frames Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <SubTitle>AV1 Video Encoding</SubTitle>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.1.0</Version>
+    <SupportedPlatforms>Linux, MacOSX, BSD</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, p7zip, cmake, perl, yasm, git</ExternalDependencies>
+    <EnvironmentSize>369</EnvironmentSize>
+    <ProjectURL>https://aomedia.googlesource.com/aom/</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/svt-av1-1.2.0/downloads.xml

@@ -0,0 +1,13 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://ultravideo.cs.tut.fi/video/Bosphorus_1920x1080_120fps_420_8bit_YUV_RAW.7z</URL>
+      <MD5>db7c7ff09acd5d7820cc4d1eb0939cf9</MD5>
+      <SHA256>e73a54088e88e6465f578625d185933e85c3209ab7105deb755a8b8918b78cab</SHA256>
+      <FileName>Bosphorus_1920x1080_120fps_420_8bit_YUV_RAW.7z</FileName>
+      <FileSize>680772328</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/svt-av1-1.2.0/install.sh

@@ -0,0 +1,16 @@
+#!/bin/sh
+
+7z x Bosphorus_1920x1080_120fps_420_8bit_YUV_RAW.7z
+
+rm -rf SVT-AV1-master
+git clone https://github.com/OpenVisualCloud/SVT-AV1.git SVT-AV1-master
+cd SVT-AV1-master/Build/linux
+./build.sh release
+echo $? > ~/install-exit-status
+
+cd ~
+
+echo "#!/bin/sh
+./SVT-AV1-master/Bin/Release/SvtAv1EncApp -i Bosphorus_1920x1080_120fps_420_8bit_YUV.yuv -w 1920 -h 1080 > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status" > svt-av1
+chmod +x svt-av1

ob-cache/test-profiles/git/svt-av1-1.2.0/results-definition.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Average Speed:		#_RESULT_# fps</OutputTemplate>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/svt-av1-1.2.0/test-definition.xml

@@ -0,0 +1,26 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>SVT-AV1</Title>
+    <AppVersion>Git</AppVersion>
+    <Description>This is a test of the Intel Open Visual Cloud Scalable Video Technology SVT-AV1 CPU-based multi-threaded video encoder for the AV1 video format with a sample 1080p YUV video file. This test profile fork builds the encoder from Git source rather than a snapshot.</Description>
+    <ResultScale>Frames Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <SubTitle>1080p 8-bit YUV To AV1 Video Encode</SubTitle>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.2.0</Version>
+    <SupportedPlatforms>Linux</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, p7zip, yasm, cmake, git</ExternalDependencies>
+    <EnvironmentSize>1900</EnvironmentSize>
+    <ProjectURL>http://github.com/OpenVisualCloud/SVT-AV1</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/svt-hevc-1.0.0/downloads.xml

@@ -0,0 +1,13 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://ultravideo.cs.tut.fi/video/Bosphorus_1920x1080_120fps_420_8bit_YUV_RAW.7z</URL>
+      <MD5>db7c7ff09acd5d7820cc4d1eb0939cf9</MD5>
+      <SHA256>e73a54088e88e6465f578625d185933e85c3209ab7105deb755a8b8918b78cab</SHA256>
+      <FileName>Bosphorus_1920x1080_120fps_420_8bit_YUV_RAW.7z</FileName>
+      <FileSize>680772328</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/svt-hevc-1.0.0/install.sh

@@ -0,0 +1,19 @@
+#!/bin/sh
+
+7z x Bosphorus_1920x1080_120fps_420_8bit_YUV_RAW.7z
+
+rm -rf SVT-HEVC-master
+git clone https://github.com/intel/SVT-HEVC.git SVT-HEVC-master
+cd SVT-HEVC-master/
+mkdir build
+cd build
+cmake ..
+make -j $NUM_CPU_CORES
+echo $? > ~/install-exit-status
+
+cd ~
+
+echo "#!/bin/sh
+./SVT-HEVC-master/Bin/Release/SvtHevcEncApp -i Bosphorus_1920x1080_120fps_420_8bit_YUV.yuv -w 1920 -h 1080 > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status" > svt-hevc
+chmod +x svt-hevc

ob-cache/test-profiles/git/svt-hevc-1.0.0/results-definition.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Average Speed:		#_RESULT_# fps</OutputTemplate>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/svt-hevc-1.0.0/test-definition.xml

@@ -0,0 +1,26 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>SVT-HEVC</Title>
+    <AppVersion>Git</AppVersion>
+    <Description>This is a test of the Intel Open Visual Cloud Scalable Video Technology SVT-HEVC CPU-based multi-threaded video encoder for the HEVC / H.265 video format with a sample 1080p YUV video file. This test uses SVT-HEVC from Git master.</Description>
+    <ResultScale>Frames Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <SubTitle>1080p 8-bit YUV To HEVC Video Encode</SubTitle>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.0.0</Version>
+    <SupportedPlatforms>Linux</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, p7zip, yasm, cmake, git</ExternalDependencies>
+    <EnvironmentSize>1900</EnvironmentSize>
+    <ProjectURL>http://github.com/intel/SVT-HEVC</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/svt-vp9-1.0.0/downloads.xml

@@ -0,0 +1,13 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://ultravideo.cs.tut.fi/video/Bosphorus_1920x1080_120fps_420_8bit_YUV_RAW.7z</URL>
+      <MD5>db7c7ff09acd5d7820cc4d1eb0939cf9</MD5>
+      <SHA256>e73a54088e88e6465f578625d185933e85c3209ab7105deb755a8b8918b78cab</SHA256>
+      <FileName>Bosphorus_1920x1080_120fps_420_8bit_YUV_RAW.7z</FileName>
+      <FileSize>680772328</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/svt-vp9-1.0.0/install.sh

@@ -0,0 +1,17 @@
+#!/bin/sh
+
+7z x Bosphorus_1920x1080_120fps_420_8bit_YUV_RAW.7z
+
+rm -rf SVT-VP9-master
+git clone https://github.com/OpenVisualCloud/SVT-VP9.git SVT-VP9-master
+cd SVT-VP9-master/Build/linux
+chmod +x build.sh
+./build.sh release
+echo $? > ~/install-exit-status
+
+cd ~
+
+echo "#!/bin/sh
+./SVT-VP9-master/Bin/Release/SvtVp9EncApp -i Bosphorus_1920x1080_120fps_420_8bit_YUV.yuv -w 1920 -h 1080 > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status" > svt-vp9
+chmod +x svt-vp9

ob-cache/test-profiles/git/svt-vp9-1.0.0/results-definition.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Average Speed:		#_RESULT_# fps</OutputTemplate>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/svt-vp9-1.0.0/test-definition.xml

@@ -0,0 +1,26 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>SVT-VP9</Title>
+    <AppVersion>Git</AppVersion>
+    <Description>This is a test of the Intel Open Visual Cloud Scalable Video Technology SVT-VP9 CPU-based multi-threaded video encoder for the VP9 video format with a sample 1080p YUV video file. This test profile uses the Git snapshot of SVT-VP9.</Description>
+    <ResultScale>Frames Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <SubTitle>1080p 8-bit YUV To VP9 Video Encode</SubTitle>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.0.0</Version>
+    <SupportedPlatforms>Linux</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, p7zip, yasm, cmake, git</ExternalDependencies>
+    <EnvironmentSize>1900</EnvironmentSize>
+    <ProjectURL>http://github.com/OpenVisualCloud/SVT-VP9</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/vpxenc-2.3.0/downloads.xml

@@ -0,0 +1,13 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://ultravideo.cs.tut.fi/video/Bosphorus_1920x1080_120fps_420_8bit_YUV_RAW.7z</URL>
+      <MD5>db7c7ff09acd5d7820cc4d1eb0939cf9</MD5>
+      <SHA256>e73a54088e88e6465f578625d185933e85c3209ab7105deb755a8b8918b78cab</SHA256>
+      <FileName>Bosphorus_1920x1080_120fps_420_8bit_YUV_RAW.7z</FileName>
+      <FileSize>680772328</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/vpxenc-2.3.0/install.sh

@@ -0,0 +1,22 @@
+#!/bin/sh
+
+git clone https://chromium.googlesource.com/webm/libvpx libvpx-master
+
+mkdir vpx
+cd libvpx-master
+
+./configure --disable-install-docs --enable-shared --prefix=$HOME/vpx
+make -j $NUM_CPU_CORES
+echo $? > ~/install-exit-status
+make install
+cd ~
+rm -rf libvpx-master
+
+7z x Bosphorus_1920x1080_120fps_420_8bit_YUV_RAW.7z
+
+echo "#!/bin/bash
+cd vpx/bin
+THREADCOUNT=\$((\$NUM_CPU_CORES>64?64:\$NUM_CPU_CORES))
+LD_PRELOAD=../lib/libvpx.so  ./vpxenc --codec=vp9 -v --threads=\$THREADCOUNT --tile-columns=6 -o /dev/null ~/Bosphorus_1920x1080_120fps_420_8bit_YUV.yuv --width=1920 --height=1080 2> \$LOG_FILE 
+echo \$? > ~/test-exit-status" > vpxenc
+chmod +x vpxenc

ob-cache/test-profiles/git/vpxenc-2.3.0/results-definition.xml

@@ -0,0 +1,12 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Pass 1/2 frame  600/601   125008B    1666b/f   50003b/s   35453 ms (#_RESULT_# fps)</OutputTemplate>
+    <LineHint> fps</LineHint>
+    <ResultBeforeString>fps</ResultBeforeString>
+    <StripFromResult>(</StripFromResult>
+    <DeleteOutputBefore>600/600</DeleteOutputBefore>
+    <DeleteOutputAfter>Pass 2</DeleteOutputAfter>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/vpxenc-2.3.0/test-definition.xml

@@ -0,0 +1,26 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>VP9 libvpx Encoding</Title>
+    <AppVersion>Git</AppVersion>
+    <Description>This is a standard video encoding performance test of Google's libvpx library and the vpxenc command for the VP9/WebM format using a sample 1080p video.</Description>
+    <ResultScale>Frames Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <SubTitle>vpxenc VP9 1080p Video Encode</SubTitle>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>2.3.0</Version>
+    <SupportedPlatforms>Linux</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, yasm, p7zip, git</ExternalDependencies>
+    <EnvironmentSize>827</EnvironmentSize>
+    <ProjectURL>http://www.webmproject.org/</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/x265-1.1.0/downloads.xml

@@ -0,0 +1,13 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://ultravideo.cs.tut.fi/video/Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z</URL>
+      <MD5>84ae521c95aa2537e16b34bbf72f2def</MD5>
+      <SHA256>e2f60b904789a60f6d1edc194d8540d401dd882e3ee3605b9b1de8feacc72133</SHA256>
+      <FileName>Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z</FileName>
+      <FileSize>676792531</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/x265-1.1.0/install.sh

@@ -0,0 +1,15 @@
+#!/bin/sh
+
+git clone https://github.com/videolan/x265 x265-master
+cd x265-master/build
+cmake ../source
+make -j $NUM_CPU_CORES
+echo $? > ~/install-exit-status
+cd ~
+
+7z x Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z
+
+echo "#!/bin/sh
+./x265-master/build/x265 Bosphorus_1920x1080_120fps_420_8bit_YUV.y4m /dev/null > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status" > x265
+chmod +x x265

ob-cache/test-profiles/git/x265-1.1.0/results-definition.xml

@@ -0,0 +1,9 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>encoded 600 frames in 55.40s #_RESULT_# fps), 1098.05 kb/s, Avg QP:34.03</OutputTemplate>
+    <LineHint>encoded 600 frames</LineHint>
+    <StripFromResult>(</StripFromResult>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/git/x265-1.1.0/test-definition.xml

@@ -0,0 +1,26 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>x265</Title>
+    <AppVersion>Git</AppVersion>
+    <Description>This is a simple test of the x265 encoder run on the CPU with a sample 1080p video file.</Description>
+    <ResultScale>Frames Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <SubTitle>H.265 1080p Video Encoding</SubTitle>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.1.0</Version>
+    <SupportedPlatforms>Linux, MacOSX, BSD</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, p7zip, cmake, yasm, Git</ExternalDependencies>
+    <EnvironmentSize>2600</EnvironmentSize>
+    <ProjectURL>http://x265.org/</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aio-stress-1.1.1/downloads.xml

@@ -0,0 +1,11 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v4.8.0 (Sokndal)-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://fsbench.filesystems.org/bench/aio-stress.c</URL>
+      <MD5>727e5fc3566837b3ea72f887d048769b</MD5>
+      <FileSize>39513</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aio-stress-1.1.1/install.sh

@@ -0,0 +1,11 @@
+#!/bin/sh
+
+cc -Wall -pthread -o aio-stress-bin aio-stress.c -laio
+echo $? > ~/install-exit-status
+
+# add support for allowing aio-test-file to be on removable media devices
+echo "#!/bin/sh
+./aio-stress-bin \$@ aio-test-file > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status
+rm -f aio-test-file" > aio-stress
+chmod +x aio-stress

ob-cache/test-profiles/pts/aio-stress-1.1.1/results-definition.xml

@@ -0,0 +1,16 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v4.8.0 (Sokndal)-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>adding stage random write
+dropping thread count to the number of contexts 1
+starting with random write
+file size 1024MB, record size 64KB, depth 64, ios per iteration 8
+max io_submit 8, buffer alignment set to 4KB
+threads 1 files 1 contexts 1 context offset 2MB verification off
+random write on aio-test-file (#_RESULT_# MB/s) 1024.00 MB in 3.84s
+thread 0 random write totals (63.36 MB/s) 1024.00 MB in 16.16s
+Running single thread version</OutputTemplate>
+    <LineHint>random write on aio-test-file</LineHint>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aio-stress-1.1.1/test-definition.xml

@@ -0,0 +1,40 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v4.8.0 (Sokndal)-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>AIO-Stress</Title>
+    <AppVersion>0.21</AppVersion>
+    <Description>AIO-Stress is an a-synchronous I/O benchmark created by SuSE. Current this profile uses a 2048MB test file and a 64KB record size.</Description>
+    <ResultScale>MB/s</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.1.1</Version>
+    <SupportedPlatforms>Linux</SupportedPlatforms>
+    <SoftwareType>Benchmark</SoftwareType>
+    <TestType>Disk</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, libaio-development</ExternalDependencies>
+    <EnvironmentSize>0.5</EnvironmentSize>
+    <EnvironmentTestingSize>1024</EnvironmentTestingSize>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Default>
+      <Arguments>-s 2g -r 64k -t 3</Arguments>
+    </Default>
+    <Option>
+      <DisplayName>Test</DisplayName>
+      <Identifier>test</Identifier>
+      <ArgumentPrefix>-o </ArgumentPrefix>
+      <Menu>
+        <Entry>
+          <Name>Random Write</Name>
+          <Value>2</Value>
+        </Entry>
+      </Menu>
+    </Option>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aircrack-ng-1.1.2/downloads.xml

@@ -0,0 +1,22 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.4.0-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>https://download.aircrack-ng.org/aircrack-ng-1.3.tar.gz</URL>
+      <MD5>c7c5b076dee0c25ee580b0f56f455623</MD5>
+      <SHA256>8ae08a7c28741f6ace2769267112053366550e7f746477081188ad38410383ca</SHA256>
+      <FileName>aircrack-ng-1.3.tar.gz</FileName>
+      <FileSize>4501507</FileSize>
+      <PlatformSpecific>Linux, BSD</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>https://download.aircrack-ng.org/aircrack-ng-1.3-win.zip</URL>
+      <MD5>319913c6e5e11aa91788ca07df40f250</MD5>
+      <SHA256>cd5f8dc621211b87e6e73b24e4d3f83ce8da7979f14bd439bd933b04be6fac2c</SHA256>
+      <FileName>aircrack-ng-1.3-win.zip</FileName>
+      <FileSize>14043368</FileSize>
+      <PlatformSpecific>Windows</PlatformSpecific>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aircrack-ng-1.1.2/install.sh

@@ -0,0 +1,24 @@
+#!/bin/sh
+tar -xf aircrack-ng-1.3.tar.gz
+cd aircrack-ng-1.3
+if [ "$OS_TYPE" = "BSD" ]
+then
+	if [ -e /usr/local/lib/libcrypto.so ]
+	then
+		env MAKE=gmake CFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib ./autogen.sh
+	else
+		env MAKE=gmake ./autogen.sh
+	fi
+	gmake -j $NUM_CPU_CORES
+else
+	./autogen.sh
+	make -j $NUM_CPU_CORES
+fi
+echo $? > ~/install-exit-status
+
+cd ~
+echo "#!/bin/sh
+cd aircrack-ng-1.3
+./src/aircrack-ng -p \$NUM_CPU_CORES \$@  2>&1 | tr '\\r' '\\n' | awk -v max=0 '{if(\$1>max){max=\$1}}END{print max \" k/s\"}' > \$LOG_FILE
+echo \$? > ~/test-exit-status" > aircrack-ng
+chmod +x aircrack-ng

ob-cache/test-profiles/pts/aircrack-ng-1.1.2/install_windows.sh

@@ -0,0 +1,8 @@
+#!/bin/sh
+unzip -o aircrack-ng-1.3-win.zip
+
+echo "#!/bin/sh
+cd aircrack-ng-1.3-win
+./bin/64bit/aircrack-ng.exe \$@  2>&1 | tr '\\r' '\\n' | awk -v max=0 '{if(\$1>max){max=\$1}}END{print max \" k/s\"}' > \$LOG_FILE
+echo \$? > ~/test-exit-status" > aircrack-ng
+chmod +x aircrack-ng

ob-cache/test-profiles/pts/aircrack-ng-1.1.2/results-definition.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.4.0-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>#_RESULT_# k/s</OutputTemplate>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aircrack-ng-1.1.2/test-definition.xml

@@ -0,0 +1,29 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.4.0-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>Aircrack-ng</Title>
+    <AppVersion>1.3</AppVersion>
+    <Description>Aircrack-ng is a tool for assessing WiFi/WLAN network security.</Description>
+    <ResultScale>k/s</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.1.2</Version>
+    <SupportedPlatforms>Linux, BSD, Windows</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, openssl-development, libtool</ExternalDependencies>
+    <EnvironmentSize>25</EnvironmentSize>
+    <ProjectURL>http://www.aircrack-ng.org/</ProjectURL>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Default>
+      <Arguments>-S -Z 30</Arguments>
+    </Default>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aobench-1.0.1/downloads.xml

@@ -0,0 +1,12 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.0.0m0-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://phoronix-test-suite.com/benchmark-files/aobench-20180207.zip</URL>
+      <MD5>6d2f7b7b82bd3363b8a5f382252588bc</MD5>
+      <SHA256>0d8fe95cb6a35042e38f87ca25f7c4f53e9d8f7d142a414be84c4439250c7a4e</SHA256>
+      <FileSize>2581</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aobench-1.0.1/install.sh

@@ -0,0 +1,10 @@
+#!/bin/sh
+
+unzip -o aobench-20180207.zip
+cc ao.c -o ao -lm -O3 $CFLAGS
+echo $? > ~/install-exit-status
+
+echo "#!/bin/sh
+./ao > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status" > aobench
+chmod +x aobench

ob-cache/test-profiles/pts/aobench-1.0.1/results-definition.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.0.0m0-->
+<PhoronixTestSuite>
+  <SystemMonitor>
+    <Sensor>sys.time</Sensor>
+  </SystemMonitor>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aobench-1.0.1/test-definition.xml

@@ -0,0 +1,24 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.0.0m0-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>AOBench</Title>
+    <Description>AOBench is a lightweight ambient occlusion renderer, written in C. The test profile is using a size of 2048 x 2048.</Description>
+    <ResultScale>Seconds</ResultScale>
+    <Proportion>LIB</Proportion>
+    <SubTitle>Size: 2048 x 2048 - Total Time</SubTitle>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.0.1</Version>
+    <SupportedPlatforms>Linux, Solaris, MacOSX, BSD, Windows</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities</ExternalDependencies>
+    <EnvironmentSize>1</EnvironmentSize>
+    <ProjectURL>https://code.google.com/archive/p/aobench/</ProjectURL>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aom-av1-1.1.0/downloads.xml

@@ -0,0 +1,20 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.0-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://ultravideo.cs.tut.fi/video/Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z</URL>
+      <MD5>84ae521c95aa2537e16b34bbf72f2def</MD5>
+      <SHA256>e2f60b904789a60f6d1edc194d8540d401dd882e3ee3605b9b1de8feacc72133</SHA256>
+      <FileName>Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z</FileName>
+      <FileSize>676792531</FileSize>
+    </Package>
+    <Package>
+      <URL>http://www.phoronix-test-suite.com/benchmark-files/aom-20190211.tar.xz</URL>
+      <MD5>c955b47dbb1292e49354f6321a09b166</MD5>
+      <SHA256>898d4e437bb3dac9cc3009bd053d7c30b5e1db41744ac95470fd831960f488f2</SHA256>
+      <FileName>aom-20190211.tar.xz</FileName>
+      <FileSize>2153100</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aom-av1-1.1.0/install.sh

@@ -0,0 +1,16 @@
+#!/bin/sh
+
+tar -xf aom-20190211.tar.xz
+cd aom-20190211/build
+cmake ..
+make -j $NUM_CPU_CORES
+echo $? > ~/install-exit-status
+cd ~
+
+7z x Bosphorus_1920x1080_120fps_420_8bit_YUV_Y4M.7z
+
+echo "#!/bin/sh
+./aom-20190211/build/aomenc -v --rt --threads=\$NUM_CPU_CORES --tile-columns=2 --limit=20 -o test.av1 Bosphorus_1920x1080_120fps_420_8bit_YUV.y4m > 1.log 2>&1
+echo \$? > ~/test-exit-status
+sed \$'s/[^[:print:]\t]/\\n/g' 1.log > \$LOG_FILE" > aom-av1
+chmod +x aom-av1

ob-cache/test-profiles/pts/aom-av1-1.1.0/results-definition.xml

@@ -0,0 +1,9 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.0-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Pass 1/2 frame    3/4        768B    2048b/f   61440b/s  165907 us #_RESULT_# fps)</OutputTemplate>
+    <LineHint>fps</LineHint>
+    <TurnCharsToSpace>(</TurnCharsToSpace>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/aom-av1-1.1.0/test-definition.xml

@@ -0,0 +1,26 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.6.0-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>AOM AV1</Title>
+    <AppVersion>2019-02-11</AppVersion>
+    <Description>This is a simple test of the AOMedia AV1 encoder run on the CPU with a sample video file.</Description>
+    <ResultScale>Frames Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <SubTitle>AV1 Video Encoding</SubTitle>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.1.0</Version>
+    <SupportedPlatforms>Linux, MacOSX, BSD</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Processor</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, p7zip, cmake, perl, yasm</ExternalDependencies>
+    <EnvironmentSize>369</EnvironmentSize>
+    <ProjectURL>https://aomedia.googlesource.com/aom/</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apache-1.7.2/downloads.xml

@@ -0,0 +1,44 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.8.1-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://archive.apache.org/dist/httpd/httpd-2.4.29.tar.bz2, http://ftp.lfs-matrix.net/pub/blfs/conglomeration/httpd/httpd-2.4.29.tar.bz2</URL>
+      <MD5>0c599404ef6b69eee95bcd9fcd094407</MD5>
+      <SHA256>777753a5a25568a2a27428b2214980564bc1c38c1abf9ccc7630b639991f7f00</SHA256>
+      <FileName>httpd-2.4.29.tar.bz2</FileName>
+      <FileSize>6567926</FileSize>
+      <PlatformSpecific>Linux, Solaris, BSD, MacOSX</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>http://www.phoronix-test-suite.com/benchmark-files/apache-ab-test-files-1.tar.gz, http://www.phoronix.net/downloads/phoronix-test-suite/benchmark-files/apache-ab-test-files-1.tar.gz</URL>
+      <MD5>ca0c5bf0dd482a2847c68a52c0759942</MD5>
+      <FileName>apache-ab-test-files-1.tar.gz</FileName>
+      <FileSize>6008</FileSize>
+    </Package>
+    <Package>
+      <URL>http://apache.melbourneitmirror.net/apr/apr-1.6.3.tar.bz2, http://archive.apache.org/dist/apr/apr-1.6.3.tar.bz2</URL>
+      <MD5>12f2a349483ad6f12db49ba01fbfdbfa</MD5>
+      <SHA256>131f06d16d7aabd097fa992a33eec2b6af3962f93e6d570a9bd4d85e95993172</SHA256>
+      <FileName>apr-1.6.3.tar.bz2</FileName>
+      <FileSize>854100</FileSize>
+      <PlatformSpecific>Linux, Solaris, BSD, MacOSX</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>http://www.apache.org/dist/apr/apr-util-1.6.1.tar.bz2, http://apache.cs.utah.edu//apr/apr-util-1.6.1.tar.bz2</URL>
+      <MD5>8ff5dc36fa39a2a3db1df196d3ed6086</MD5>
+      <SHA256>d3e12f7b6ad12687572a3a39475545a072608f4ba03a6ce8a3778f607dd0035b</SHA256>
+      <FileName>apr-util-1.6.1.tar.bz2</FileName>
+      <FileSize>428595</FileSize>
+      <PlatformSpecific>Linux, Solaris, BSD, MacOSX</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>http://phoronix-test-suite.com/benchmark-files/Apache24-2.4.29-x64-vc14-r2-ah.zip</URL>
+      <MD5>8f7e773c8db85d55e4cc81ef153fef61</MD5>
+      <SHA256>1e6f26e8fbe3decd4c31cff3347a074c7db1f8af5ac71bf4386a1e80304e47ef</SHA256>
+      <FileName>Apache24-2.4.29-x64-vc14-r2-ah.zip</FileName>
+      <FileSize>10444082</FileSize>
+      <PlatformSpecific>Windows</PlatformSpecific>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apache-1.7.2/install.sh

@@ -0,0 +1,29 @@
+#!/bin/sh
+
+mkdir $HOME/httpd_
+
+tar -zxvf apache-ab-test-files-1.tar.gz
+tar -jxvf httpd-2.4.29.tar.bz2
+tar -jxvf apr-util-1.6.1.tar.bz2
+tar -jxvf apr-1.6.3.tar.bz2
+mv apr-1.6.3 httpd-2.4.29/srclib/apr
+mv apr-util-1.6.1 httpd-2.4.29/srclib/apr-util
+
+cd httpd-2.4.29/
+./configure --prefix=$HOME/httpd_ --with-included-apr
+make -j $NUM_CPU_JOBS
+echo $? > ~/install-exit-status
+make install
+cd ~
+rm -rf httpd-2.4.29/
+rm -rf httpd_/manual/
+
+patch -p0 < CHANGE-APACHE-PORT.patch
+mv -f test.html httpd_/htdocs/
+mv -f pts.png httpd_/htdocs/
+
+echo "#!/bin/sh
+./httpd_/bin/ab \$@ > \$LOG_FILE 2>&1
+echo \$? > ~/test-exit-status" > apache
+
+chmod +x apache

ob-cache/test-profiles/pts/apache-1.7.2/install_windows.sh

@@ -0,0 +1,19 @@
+#!/bin/bash
+
+unzip -o Apache24-2.4.29-x64-vc14-r2-ah.zip
+
+tar -xf apache-ab-test-files-1.tar.gz
+mv -f test.html Apache24/htdocs/
+mv -f pts.png Apache24/htdocs/
+
+cd Apache24/conf
+tail -n +2 httpd.conf > httpd.conf.2
+echo "Define SRVROOT \"$DEBUG_HOME\Apache24\"
+" > httpd.conf
+cat httpd.conf.2 >> httpd.conf
+cd ~
+
+echo "#!/bin/sh
+cd Apache24/bin
+./ab.exe \$@ > \$LOG_FILE" > apache
+chmod +x apache

ob-cache/test-profiles/pts/apache-1.7.2/post.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+./httpd_/bin/apachectl -k stop
+rm -f httpd_/logs/*
+sleep 3

ob-cache/test-profiles/pts/apache-1.7.2/post_windows.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+kill -TERM `cat $HOME/Apache24/logs/httpd.pid`
+sleep 10
+rm -f $HOME/Apache24/logs/*

ob-cache/test-profiles/pts/apache-1.7.2/pre.sh

@@ -0,0 +1,3 @@
+#!/bin/sh
+./httpd_/bin/apachectl -k start -f $HOME/httpd_/conf/httpd.conf
+sleep 5

ob-cache/test-profiles/pts/apache-1.7.2/pre_windows.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+cd Apache24/bin
+./httpd.exe &
+sleep 10

ob-cache/test-profiles/pts/apache-1.7.2/results-definition.xml

@@ -0,0 +1,20 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.8.1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Document Path:          /test.html
+Document Length:        3064 bytes
+
+Concurrency Level:      100
+Time taken for tests:   29.224 seconds
+Complete requests:      500000
+Failed requests:        0
+Write errors:           0
+Total transferred:      1659096222 bytes
+HTML transferred:       1532088856 bytes
+Requests per second:    #_RESULT_# [#/sec] (mean)
+Time per request:       5.845 [ms] (mean)
+Time per request:       0.058 [ms] (mean, across all concurrent requests)
+Transfer rate:          55440.49 [Kbytes/sec] received</OutputTemplate>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apache-1.7.2/test-definition.xml

@@ -0,0 +1,31 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.8.1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>Apache Benchmark</Title>
+    <AppVersion>2.4.29</AppVersion>
+    <Description>This is a test of ab, which is the Apache benchmark program. This test profile measures how many requests per second a given system can sustain when carrying out 1,000,000 requests with 100 requests being carried out concurrently.</Description>
+    <ResultScale>Requests Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <SubTitle>Static Web Page Serving</SubTitle>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.7.2</Version>
+    <SupportedPlatforms>Linux, Solaris, BSD, MacOSX, Windows</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>System</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, perl, pcre, expat</ExternalDependencies>
+    <EnvironmentSize>208</EnvironmentSize>
+    <ProjectURL>http://www.apache.org/</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Default>
+      <Arguments>-n 1000000 -c 100 http://localhost:8088/test.html</Arguments>
+    </Default>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apache-siege-1.0.4/downloads.xml

@@ -0,0 +1,44 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.8.1-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://archive.apache.org/dist/httpd/httpd-2.4.29.tar.bz2, http://ftp.lfs-matrix.net/pub/blfs/conglomeration/httpd/httpd-2.4.29.tar.bz2</URL>
+      <MD5>0c599404ef6b69eee95bcd9fcd094407</MD5>
+      <SHA256>777753a5a25568a2a27428b2214980564bc1c38c1abf9ccc7630b639991f7f00</SHA256>
+      <FileName>httpd-2.4.29.tar.bz2</FileName>
+      <FileSize>6567926</FileSize>
+      <PlatformSpecific>Linux, Solaris, BSD, MacOSX</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>http://www.phoronix-test-suite.com/benchmark-files/apache-ab-test-files-1.tar.gz, http://www.phoronix.net/downloads/phoronix-test-suite/benchmark-files/apache-ab-test-files-1.tar.gz</URL>
+      <MD5>ca0c5bf0dd482a2847c68a52c0759942</MD5>
+      <FileName>apache-ab-test-files-1.tar.gz</FileName>
+      <FileSize>6008</FileSize>
+    </Package>
+    <Package>
+      <URL>http://apache.melbourneitmirror.net/apr/apr-1.6.3.tar.bz2, http://archive.apache.org/dist/apr/apr-1.6.3.tar.bz2</URL>
+      <MD5>12f2a349483ad6f12db49ba01fbfdbfa</MD5>
+      <SHA256>131f06d16d7aabd097fa992a33eec2b6af3962f93e6d570a9bd4d85e95993172</SHA256>
+      <FileName>apr-1.6.3.tar.bz2</FileName>
+      <FileSize>854100</FileSize>
+      <PlatformSpecific>Linux, Solaris, BSD, MacOSX</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>http://www.apache.org/dist/apr/apr-util-1.6.1.tar.bz2, http://apache.cs.utah.edu//apr/apr-util-1.6.1.tar.bz2</URL>
+      <MD5>8ff5dc36fa39a2a3db1df196d3ed6086</MD5>
+      <SHA256>d3e12f7b6ad12687572a3a39475545a072608f4ba03a6ce8a3778f607dd0035b</SHA256>
+      <FileName>apr-util-1.6.1.tar.bz2</FileName>
+      <FileSize>428595</FileSize>
+      <PlatformSpecific>Linux, Solaris, BSD, MacOSX</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>http://download.joedog.org/siege/siege-3.1.4.tar.gz</URL>
+      <MD5>a893b92a453e1c1d596314aa951981f4</MD5>
+      <SHA256>0436970ca81ade76eb419bc42d3f8460b475079eb5d8f199d2fa83328893099d</SHA256>
+      <FileName>siege-3.1.4.tar.gz</FileName>
+      <FileSize>530149</FileSize>
+      <PlatformSpecific>Linux, Solaris, BSD, MacOSX</PlatformSpecific>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apache-siege-1.0.4/install.sh

@@ -0,0 +1,40 @@
+#!/bin/sh
+
+mkdir $HOME/httpd_
+
+tar -zxvf apache-ab-test-files-1.tar.gz
+tar -jxvf httpd-2.4.29.tar.bz2
+tar -jxvf apr-util-1.6.1.tar.bz2
+tar -jxvf apr-1.6.3.tar.bz2
+mv apr-1.6.3 httpd-2.4.29/srclib/apr
+mv apr-util-1.6.1 httpd-2.4.29/srclib/apr-util
+
+cd httpd-2.4.29/
+./configure --prefix=$HOME/httpd_ --with-included-apr
+make -j $NUM_CPU_CORES
+echo $? > ~/install-exit-status
+make install
+cd ~
+rm -rf httpd-2.4.29/
+rm -rf httpd_/manual/
+
+patch -p0 < CHANGE-APACHE-PORT.patch
+mv -f test.html httpd_/htdocs/
+mv -f pts.png httpd_/htdocs/
+
+
+cd ~
+tar -xf siege-3.1.4.tar.gz
+cd siege-3.1.4
+./configure
+make -j $NUM_CPU_CORES
+cd utils
+bash siege.config
+cd ~
+
+echo "#!/bin/sh
+cd siege-3.1.4/src
+./siege \$@ 2>&1 | grep -v HTTP > \$LOG_FILE
+echo \$? > ~/test-exit-status" > apache-siege
+
+chmod +x apache-siege

ob-cache/test-profiles/pts/apache-siege-1.0.4/post.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+./httpd_/bin/apachectl -k stop
+rm -f httpd_/logs/*
+sleep 3

ob-cache/test-profiles/pts/apache-siege-1.0.4/pre.sh

@@ -0,0 +1,7 @@
+#!/bin/sh
+./httpd_/bin/apachectl -k start -f $HOME/httpd_/conf/httpd.conf
+sleep 5
+
+cd siege-3.1.4/utils
+bash siege.config
+cd ~

ob-cache/test-profiles/pts/apache-siege-1.0.4/results-definition.xml

@@ -0,0 +1,7 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.8.1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>Transaction rate:	    #_RESULT_# trans/sec</OutputTemplate>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apache-siege-1.0.4/test-definition.xml

@@ -0,0 +1,62 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.8.1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>Apache Siege</Title>
+    <AppVersion>2.4.29</AppVersion>
+    <Description>This is a test of the Apache web server performance being facilitated by the Siege web serverb enchmark program.</Description>
+    <ResultScale>Transactions Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.0.4</Version>
+    <SupportedPlatforms>Linux, Solaris, BSD, MacOSX</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>System</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, perl, pcre, openssl-development, expat</ExternalDependencies>
+    <EnvironmentSize>208</EnvironmentSize>
+    <ProjectURL>https://www.joedog.org/siege-home/</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Default>
+      <Arguments>-r10000 -d1 -b </Arguments>
+      <PostArguments> http://localhost:8088/test.html</PostArguments>
+    </Default>
+    <Option>
+      <DisplayName>Concurrent Users</DisplayName>
+      <Identifier>concurrent</Identifier>
+      <ArgumentPrefix>-c</ArgumentPrefix>
+      <Menu>
+        <Entry>
+          <Name>1</Name>
+          <Value>1</Value>
+        </Entry>
+        <Entry>
+          <Name>10</Name>
+          <Value>10</Value>
+        </Entry>
+        <Entry>
+          <Name>50</Name>
+          <Value>50</Value>
+        </Entry>
+        <Entry>
+          <Name>100</Name>
+          <Value>100</Value>
+        </Entry>
+        <Entry>
+          <Name>200</Name>
+          <Value>200</Value>
+        </Entry>
+        <Entry>
+          <Name>250</Name>
+          <Value>250</Value>
+        </Entry>
+      </Menu>
+    </Option>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apitest-1.1.0/downloads.xml

@@ -0,0 +1,12 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v5.2.1 (Khanino)-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://phoronix-test-suite.com/benchmark-files/apitest-20140726.tar.bz2</URL>
+      <MD5>28ac817817d35ab335ad40dcec682b6d</MD5>
+      <SHA256>67a48803c10c147f99632e1aa54cf1afce1b785d2ecf850b5f960e72d2dd5258</SHA256>
+      <FileSize>23855194</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apitest-1.1.0/install.sh

@@ -0,0 +1,16 @@
+#!/bin/sh
+
+tar -xjf apitest-20140726.tar.bz2
+cd apitest-master
+mkdir out
+cd out
+cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release -DHAVE_LIBUDEV_H=0  ..
+make -j $NUM_CPU_JOBS
+echo $? > ~/install-exit-status
+
+cd ~
+echo "#!/bin/sh
+cd apitest-master/bin/
+./apitest \$@ > \$LOG_FILE 2>&1
+echo $? > ~/test-exit-status" > apitest
+chmod +x apitest

ob-cache/test-profiles/pts/apitest-1.1.0/results-definition.xml

@@ -0,0 +1,8 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v5.2.1 (Khanino)-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate> NullProblem             oglcore    NullSolution                     30316       15.000     #_RESULT_#        0.495</OutputTemplate>
+    <LineBeforeHint>fps</LineBeforeHint>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/apitest-1.1.0/test-definition.xml

@@ -0,0 +1,137 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v5.2.1 (Khanino)-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>APITest</Title>
+    <AppVersion>2014-07-26</AppVersion>
+    <Description>APITest is a micro-benchmark developed by John McDonald of OpenGL 4 functionality.</Description>
+    <ResultScale>Frames Per Second</ResultScale>
+    <Proportion>HIB</Proportion>
+    <TimesToRun>3</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.1.0</Version>
+    <SupportedPlatforms>Linux, MacOSX</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>Graphics</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <ExternalDependencies>build-utilities, sdl2-development, cmake</ExternalDependencies>
+    <EnvironmentSize>225</EnvironmentSize>
+    <ProjectURL>http://github.com/nvMcJohn/apitest</ProjectURL>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Default>
+      <Arguments>-b -t 10 -a oglcore</Arguments>
+    </Default>
+    <Option>
+      <DisplayName>Resolution</DisplayName>
+      <Identifier>auto-resolution</Identifier>
+      <ArgumentPrefix>-r </ArgumentPrefix>
+      <DefaultEntry>0</DefaultEntry>
+      <Menu>
+        <Entry>
+          <Name>$VIDEO_WIDTH x $VIDEO_HEIGHT</Name>
+          <Value>$VIDEO_WIDTH $VIDEO_HEIGHT</Value>
+        </Entry>
+      </Menu>
+    </Option>
+    <Option>
+      <DisplayName>Test</DisplayName>
+      <Identifier>test</Identifier>
+      <ArgumentPrefix>-s </ArgumentPrefix>
+      <DefaultEntry>0</DefaultEntry>
+      <Menu>
+        <Entry>
+          <Name>DynamicStreaming GLBufferSubData</Name>
+          <Value>GLBufferSubData</Value>
+        </Entry>
+        <Entry>
+          <Name>DynamicStreaming GLMapUnsynchronized</Name>
+          <Value>GLMapUnsynchronized</Value>
+        </Entry>
+        <Entry>
+          <Name>DynamicStreaming GLMapPersistent</Name>
+          <Value>GLMapPersistent</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLUniform</Name>
+          <Value>GLUniform</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLDrawLoop</Name>
+          <Value>GLDrawLoop</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLMultiDrawBuffer-SDP</Name>
+          <Value>GLMultiDrawBuffer-SDP</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLMultiDrawBuffer-NoSDP</Name>
+          <Value>GLMultiDrawBuffer-NoSDP</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLBufferRange</Name>
+          <Value>GLBufferRange</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLBufferSubData</Name>
+          <Value>GLBufferSubData</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLBufferStorage-SDP</Name>
+          <Value>GLBufferStorage-SDP</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLBufferStorage-NoSDP</Name>
+          <Value>GLBufferStorage-NoSDP</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLDynamicBuffer</Name>
+          <Value>GLDynamicBuffer</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLMapUnsynchronized</Name>
+          <Value>GLMapUnsynchronized</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLMapPersistent</Name>
+          <Value>GLMapPersistent</Value>
+        </Entry>
+        <Entry>
+          <Name>UntexturedObjects GLTexCoord</Name>
+          <Value>GLTexCoord</Value>
+        </Entry>
+        <Entry>
+          <Name>TexturedQuadsProblem GLBindless</Name>
+          <Value>GLBindless</Value>
+        </Entry>
+        <Entry>
+          <Name>TexturedQuadsProblem GLNaive</Name>
+          <Value>GLNaive</Value>
+        </Entry>
+        <Entry>
+          <Name>TexturedQuadsProblem GLNaiveUniform</Name>
+          <Value>GLNaiveUniform</Value>
+        </Entry>
+        <Entry>
+          <Name>TexturedQuadsProblem GLNoTex</Name>
+          <Value>GLNoTex</Value>
+        </Entry>
+        <Entry>
+          <Name>TexturedQuadsProblem GLNoTexUniform</Name>
+          <Value>GLNoTexUniform</Value>
+        </Entry>
+        <Entry>
+          <Name>TexturedQuadsProblem GLSBTA</Name>
+          <Value>GLSBTA</Value>
+        </Entry>
+        <Entry>
+          <Name>TexturedQuadsProblem GLTextureArray</Name>
+          <Value>GLTextureArray</Value>
+        </Entry>
+      </Menu>
+    </Option>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/appleseed-1.0.0/downloads.xml

@@ -0,0 +1,43 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.8.1-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>https://github.com/appleseedhq/appleseed/releases/download/2.0.0-beta/appleseed-2.0.0-beta-0-g5cff7b96b-linux64-gcc48.zip</URL>
+      <MD5>092da084b0c6f0b4871e478ae308aa8c</MD5>
+      <SHA256>4cc73c2c836c153dd9056ccf6499738923e826dbebd8652776ff3c7973f824f2</SHA256>
+      <FileName>appleseed-2.0.0-beta-0-g5cff7b96b-linux64-gcc48.zip</FileName>
+      <FileSize>108932072</FileSize>
+      <PlatformSpecific>Linux</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>https://github.com/appleseedhq/appleseed/releases/download/2.0.0-beta/appleseed-2.0.0-beta-0-g5cff7b96b-win64-vs140.zip</URL>
+      <MD5>4e6befb880dfeece518f4ccadd5f2f2c</MD5>
+      <SHA256>1cc690aac45100b3e228cf6888cec9faf777994ef32ed96a3110d0b0e102bb57</SHA256>
+      <FileName>appleseed-2.0.0-beta-0-g5cff7b96b-win64-vs140.zip</FileName>
+      <FileSize>62994385</FileSize>
+      <PlatformSpecific>Windows</PlatformSpecific>
+    </Package>
+    <Package>
+      <URL>https://github.com/appleseedhq/appleseed-scenes/releases/download/1.8.0-beta/emily_1.4.zip</URL>
+      <MD5>ac70fa83eb5f12966fd28a000c54c210</MD5>
+      <SHA256>3ad9332ad0777d70871454228587372fee9ad98db35483cd25c96b7663b81886</SHA256>
+      <FileName>emily_1.4.zip</FileName>
+      <FileSize>417182051</FileSize>
+    </Package>
+    <Package>
+      <URL>https://github.com/appleseedhq/appleseed-scenes/releases/download/1.8.0-beta/disney_material_1.2.zip</URL>
+      <MD5>4eebe5b47792b8ca031995cf9e613893</MD5>
+      <SHA256>6aa882aab4ce14129fbc899a9e00e65d281a239373f17705d536488511d719c4</SHA256>
+      <FileName>disney_material_1.2.zip</FileName>
+      <FileSize>3436200</FileSize>
+    </Package>
+    <Package>
+      <URL>https://github.com/appleseedhq/appleseed-scenes/releases/download/1.8.0-beta/material_tester_1.4.zip</URL>
+      <MD5>64cfa21fad171fa1ceb32539b448755a</MD5>
+      <SHA256>0e185ed1363f98c826c7b37763ccac0b203118df850c9583be29299b4f8da22e</SHA256>
+      <FileName>material_tester_1.4.zip</FileName>
+      <FileSize>21341121</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/appleseed-1.0.0/install.sh

@@ -0,0 +1,15 @@
+#!/bin/sh
+
+unzip -o appleseed-2.0.0-beta-0-g5cff7b96b-linux64-gcc48.zip
+unzip -o emily_1.4.zip
+unzip -o disney_material_1.2.zip
+unzip -o material_tester_1.4.zip
+
+cp -va emily/* appleseed
+cp -va disney_material/* appleseed
+cp -va material_tester/* appleseed
+
+echo "#!/bin/bash
+cd appleseed
+./bin/appleseed.cli --benchmark-mode \$@ > \$LOG_FILE 2>&1" > appleseed-benchmark
+chmod +x appleseed-benchmark

ob-cache/test-profiles/pts/appleseed-1.0.0/install_windows.sh

@@ -0,0 +1,15 @@
+#!/bin/sh
+
+unzip -o appleseed-2.0.0-beta-0-g5cff7b96b-win64-vs140.zip
+unzip -o emily_1.4.zip
+unzip -o disney_material_1.2.zip
+unzip -o material_tester_1.4.zip
+
+cp -va emily/* appleseed
+cp -va disney_material/* appleseed
+cp -va material_tester/* appleseed
+
+echo "#!/bin/bash
+cd appleseed
+./bin/appleseed.cli.exe --benchmark-mode \$@ > \$LOG_FILE 2>&1" > appleseed-benchmark
+chmod +x appleseed-benchmark

ob-cache/test-profiles/pts/appleseed-1.0.0/results-definition.xml

@@ -0,0 +1,8 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.8.1-->
+<PhoronixTestSuite>
+  <ResultsParser>
+    <OutputTemplate>render_time=#_RESULT_#</OutputTemplate>
+    <LineHint>render_time</LineHint>
+  </ResultsParser>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/appleseed-1.0.0/test-definition.xml

@@ -0,0 +1,46 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.8.1-->
+<PhoronixTestSuite>
+  <TestInformation>
+    <Title>Appleseed</Title>
+    <AppVersion>2.0 Beta</AppVersion>
+    <Description>Appleseed is an open-source production renderer focused on physically-based global illumination rendering engine primarily designed for animation and visual effects.</Description>
+    <ResultScale>Seconds</ResultScale>
+    <Proportion>LIB</Proportion>
+    <Executable>appleseed-benchmark</Executable>
+    <TimesToRun>1</TimesToRun>
+  </TestInformation>
+  <TestProfile>
+    <Version>1.0.0</Version>
+    <SupportedPlatforms>Linux, Windows</SupportedPlatforms>
+    <SoftwareType>Utility</SoftwareType>
+    <TestType>System</TestType>
+    <License>Free</License>
+    <Status>Verified</Status>
+    <SupportedArchitectures>x86_64</SupportedArchitectures>
+    <EnvironmentSize>800</EnvironmentSize>
+    <ProjectURL>https://appleseedhq.net/</ProjectURL>
+    <InternalTags>SMP</InternalTags>
+    <Maintainer>Michael Larabel</Maintainer>
+  </TestProfile>
+  <TestSettings>
+    <Option>
+      <DisplayName>Scene</DisplayName>
+      <Identifier>scene</Identifier>
+      <Menu>
+        <Entry>
+          <Name>Emily</Name>
+          <Value>emily.appleseed</Value>
+        </Entry>
+        <Entry>
+          <Name>Disney Material</Name>
+          <Value>disney_material_1.appleseed</Value>
+        </Entry>
+        <Entry>
+          <Name>Material Tester</Name>
+          <Value>material_tester_ambient_occlusion.appleseed</Value>
+        </Entry>
+      </Menu>
+    </Option>
+  </TestSettings>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/arrayfire-1.0.1/downloads.xml

@@ -0,0 +1,12 @@
+<?xml version="1.0"?>
+<!--Phoronix Test Suite v8.0.0m2-->
+<PhoronixTestSuite>
+  <Downloads>
+    <Package>
+      <URL>http://phoronix-test-suite.com/benchmark-files/arrayfire-benchmark-git-20170116.tar.gz</URL>
+      <MD5>a221ca234f4188c7c43a7a776a5f1aa3</MD5>
+      <SHA256>26df747a47543946c72f0d5d309620a6998894e45fe81fe2aecac696764fab34</SHA256>
+      <FileSize>7431207</FileSize>
+    </Package>
+  </Downloads>
+</PhoronixTestSuite>

ob-cache/test-profiles/pts/arrayfire-1.0.1/install.sh

@@ -0,0 +1,82 @@
+#!/bin/sh
+
+# Not all of these dependencies below may be covered automatically by PTS, in particular arrayfire.
+# http://arrayfire.com/download-splash/?redirect_to=/download or compile from source from gitub:
+# https://github.com/arrayfire/arrayfire
+# sudo apt-get install --no-install-recommends libboost-all-dev
+
+#procuring arrayfire is easy!  We prefer the no-gl version so we can run headless
+# VER=3.4.2; wget http://arrayfire.com/installer_archive/${VER}/ArrayFire-no-gl-v${VER}_Linux_x86_64.sh
+#ArrayFire is expected to be installed to /opt
+#to install via the shell package ArrayFire supplies - use the prefix option ./ArrayFire-v3.4.2_Linux_x86_64.sh --prefix=/opt
+#when prompted with "Do you want to include the subdirectory arrayfire-3?", say yes
+
+#launch with the environment variable OpenCL_INCLUDE_DIR set to the path to cl.h
+#e.g. OpenCL_INCLUDE_DIR=/opt/rocm/opencl/include/CL/ phoronix-test-suite install arrayfire
+#also make sure to use the correct path to your OpenCL shared library
+#e.g. LD_LIBRARY_PATH=/opt/rocm/opencl/lib/x86_64 phoronix-test-suite run arrayfire
+
+if [ -d /usr/local/cuda ]
+then
+    PATH="/usr/local/cuda/bin:$PATH"
+    export LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/cuda/nvvm/lib64:$LD_LIBRARY_PATH
+    export CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda
+    if [ -n OpenCL_INCLUDE_DIR ]
+    then
+        export OpenCL_INCLUDE_DIR=/usr/local/cuda/include
+    fi
+else
+    if [ -d /opt/rocm/opencl/lib/x86_64 ]
+    then
+        export OpenCL_INCLUDE_DIR=/opt/rocm/opencl/lib/x86_64
+    elif [ -n OpenCL_INCLUDE_DIR ]
+    then
+        export OpenCL_INCLUDE_DIR=/usr/include
+    fi
+fi
+
+rm -rf arrayfire-benchmark.git
+tar -xvzf arrayfire-benchmark-git-20170116.tar.gz
+
+if [ ! -e arrayfire-benchmark.git/bin/benchmark_opencl ]
+then
+    pushd arrayfire-benchmark.git
+    AF_ROOT=`pwd`
+    ./updateLibraries.sh --build --prefix=${AF_ROOT} --cl-path=${OpenCL_INCLUDE_DIR}
+    pushd build
+    cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DArrayFire_DIR=${AF_ROOT}/arrayfire-3/share/ArrayFire/cmake -DOpenCL_INCLUDE_DIR=${OpenCL_INCLUDE_DIR} ..
+    #make -j${NUM_CPU_CORES} benchmark_opencl
+    make -j${NUM_CPU_CORES}
+    echo $? > ~/install-exit-status
+    popd
+    popd
+fi
+#pushd arrayfire-benchmark.git
+cd ~/
+echo "#!/bin/sh
+cd arrayfire-benchmark.git
+if [ -d /usr/local/cuda ]
+then
+    export LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/cuda/nvvm/lib64:\$LD_LIBRARY_PATH
+fi
+if [ -d /opt/rocm/opencl/lib/x86_64 ]
+then
+    export LD_LIBRARY_PATH=/opt/rocm/opencl/lib/x86_64:\$LD_LIBRARY_PATH
+fi
+if [ -n COMPUTE_DEVICE ]
+then
+   export COMPUTE_DEVICE=0
+fi
+timeout -s SIGKILL --preserve-status 30 ./\$@ -d \${COMPUTE_DEVICE} > \$LOG_FILE
+echo \$? > ~/test-exit-status
+perl -pi -e 's/\e\[(\d+)(;\d+)*m//g' \$LOG_FILE  #scrub ansi codes
+awk 'NR > 4 { print \$0 } NR <= 4 { next }' \${LOG_FILE} > \${LOG_FILE}.tmp; mv \${LOG_FILE}.tmp \${LOG_FILE}
+grep -v Baseline \${LOG_FILE} > \${LOG_FILE}.tmp; mv \${LOG_FILE}.tmp \${LOG_FILE}
+
+perl -pi -e 's/\e\[(\d+)(;\d+)*m//g' \$LOG_FILE  #scrub ansi codes
+sed -n '/Celero/,\$p' \$LOG_FILE
+awk '/Celero/,0' \$LOG_FILE
+awk 'NR > 4 { print \$0 } NR <= 4 { next }' \$LOG_FILE
+grep -v Baseline \${LOG_FILE} > \${LOG_FILE}.tmp; mv \${LOG_FILE}.tmp \${LOG_FILE}
+" > arrayfire
+chmod +x arrayfire