Understanding PBench Output

NOTE: incomplete draft, do not read or review

cd Downloads

./pbench/pbench_darwin_arm64 run -s http://localhost:8080 -o pbench3 pbench/benchmarks/tpc-ds/ds_power.json pbench/benchmarks/tpc-ds/sf1.json

Delete this note before publishing.

NOTE: I am not sure Understanding PBench Output is the best name for this page. Think about a better one?

Delete this note before publishing.

NOTE: to do:

• edit pbench/benchmarks/tpc-ds/ds_power.json pbench/benchmarks/tpc-ds/sf1.json to add "save_column_metadata": true, run command, find file example

• add cold and warm run parameters to get example filenames that show [c|w] in the filename

Delete this note before publishing.

Pbench creates a set of files for each benchmark run. This page describes where to find the PBench output files, what kinds of files are created by PBench, and how PBench defines the file names.

PBench Output File Location

A benchmark run creates files in an output directory and a subdirectory called a run directory.

Output Directory

Output files are saved to the output directory. Use pbench run -o to set an output directory.

If the directory specified by -o does not exist, it is created.

If -o is not used, the default output directory is the current directory.

Each benchmark run saves the files for that run in a run directory in the output directory.

Run Directory

Each run creates a run directory in the output directory. Use pbench run -n to set a a run directory name.

The run directory name is either:

• <main stage name>_<current time> (default)
• the name assigned with -n

The <main stage name> is the names of the stage files separated by underscores.

Main Stage Name Examples

• ./pbench run -s http://localhost:8080 pbench/benchmarks/tpc-ds/sf1.json pbench/benchmarks/tpc-ds/ds_rand5.json results in:

  • main stage name sf1_ds_rand5
  • run directory sf1_ds_rand5_<current time>

• ./pbench/pbench -s http://localhost:8080 pbench/benchmarks/tpc-ds/ds_rand5.json pbench/benchmarks/tpc-ds/sf1.json results in:

  • main stage name ds_rand5_sf1
  • run directory of ds_rand5_sf1_<current time>

Output and Run Directory Examples

• pbench run saves the output to <current directory>/<main stage name>-<current time>, because of the defaults for -o and -n.
• pbench run -o pbench_results saves the output to the directory pbench_results/<main stage name>-<current time>.
• pbench run -n today saves the output to <current directory>/today.
• pbench run -o pbench_results -n today saves the output to pbench_results/today.

PBench Output Files

All PBench output files are saved into the run directory in the output directory.

Log file

ds_power_sf1_240509-130537.log

Output JSON File

sf1_query_01.json

Error JSON File

sf1_query_02.error.json

Summary File

sf1_summary.csv

Column Metadata File

sf1_query_01.cols.json

Query Result File Name

The query result file name of a file generated by "save_output": true is created using the following format:

[query_file]_[query_index]_[c|w][sequence_number]

• [query_file] - One of the two following, depending on if the query is in queries or query_files:

  • queries in the queries array are displayed inline
  • queries in a query file display the file name without extension

• [query_index] - The query's position in the queries array, or the position in the file named in query_files.

  The first query is assigned a query_index of 0.

  NOTE: [query_index] is omitted if the queries array, or the file named in query_files, only has one query.

• [c|w] - Identifies the run as a cold run or a warm run.

  • c - cold
  • w - warm

• [sequence_number] - The sequence number starts at 0. For example, if "cold_runs": 2 and "warm_runs": 3, then:

  • the sequence is 0-4
  • 0, 1 are the cold runs
  • 2, 3, 4 are the warm runs