Performance Regression or Improvement: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs:mean_load_model_latency_milli_secs

[github-actions]

Performance change found in the
test: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs for the metric: mean_load_model_latency_milli_secs.

For more information on how to triage the alerts, please look at
Triage performance alert issues section of the README.

Test description: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU.
Test link -

beam/.test-infra/jenkins/job_InferenceBenchmarkTests_Python.groovy

Line 151 in 42d0a6e

test : 'apache_beam.testing.benchmarks.inference.pytorch_image_classification_benchmarks',

Test dashboard - http://metrics.beam.apache.org/d/ZpS8Uf44z/python-ml-runinference-benchmarks?orgId=1&viewPanel=7

timestamp: Tue Jan 21 06:36:10 2025, metric_value: 120691.52
timestamp: Mon Jan 20 06:41:34 2025, metric_value: 115474.82
timestamp: Sun Jan 19 06:41:50 2025, metric_value: 116304.58 <---- Anomaly
timestamp: Sat Jan 18 06:59:57 2025, metric_value: 124728.72
timestamp: Fri Jan 17 06:38:08 2025, metric_value: 124745.48
timestamp: Thu Jan 16 06:37:10 2025, metric_value: 129002.75
timestamp: Wed Jan 15 06:41:40 2025, metric_value: 128322.32
timestamp: Tue Jan 14 06:39:24 2025, metric_value: 126983.60
timestamp: Mon Jan 13 06:34:22 2025, metric_value: 123951.52
timestamp: Sun Jan 12 06:42:25 2025, metric_value: 124889.77
timestamp: Sat Jan 11 06:49:59 2025, metric_value: 130968.50
timestamp: Fri Jan 10 06:39:12 2025, metric_value: 129227.57
timestamp: Thu Jan  9 06:35:49 2025, metric_value: 131263.98

[github-actions]

Performance change found in the
test: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs for the metric: mean_load_model_latency_milli_secs.

For more information on how to triage the alerts, please look at
Triage performance alert issues section of the README.

Test description: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU.
Test link -

beam/.test-infra/jenkins/job_InferenceBenchmarkTests_Python.groovy

Line 151 in 42d0a6e

test : 'apache_beam.testing.benchmarks.inference.pytorch_image_classification_benchmarks',

Test dashboard - http://metrics.beam.apache.org/d/ZpS8Uf44z/python-ml-runinference-benchmarks?orgId=1&viewPanel=7

timestamp: Mon Feb  3 06:14:52 2025, metric_value: 120352.85
timestamp: Sun Feb  2 06:25:12 2025, metric_value: 123105.87
timestamp: Sat Feb  1 06:19:08 2025, metric_value: 122917.00
timestamp: Fri Jan 31 06:15:15 2025, metric_value: 125189.10
timestamp: Wed Jan 29 06:34:31 2025, metric_value: 119985.23
timestamp: Tue Jan 28 06:32:22 2025, metric_value: 123488.70
timestamp: Mon Jan 27 06:15:54 2025, metric_value: 120767.65
timestamp: Sun Jan 26 06:17:44 2025, metric_value: 121232.82
timestamp: Sat Jan 25 06:37:00 2025, metric_value: 123815.07
timestamp: Fri Jan 24 06:29:51 2025, metric_value: 122248.15
timestamp: Thu Jan 23 06:20:08 2025, metric_value: 123471.17 <---- Anomaly
timestamp: Wed Jan 22 06:41:47 2025, metric_value: 118871.80
timestamp: Tue Jan 21 06:36:10 2025, metric_value: 120691.52
timestamp: Mon Jan 20 06:41:34 2025, metric_value: 115474.82
timestamp: Sun Jan 19 06:41:50 2025, metric_value: 116304.58
timestamp: Sat Jan 18 06:59:57 2025, metric_value: 124728.72
timestamp: Fri Jan 17 06:38:08 2025, metric_value: 124745.48
timestamp: Thu Jan 16 06:37:10 2025, metric_value: 129002.75
timestamp: Wed Jan 15 06:41:40 2025, metric_value: 128322.32
timestamp: Tue Jan 14 06:39:24 2025, metric_value: 126983.60
timestamp: Mon Jan 13 06:34:22 2025, metric_value: 123951.52

[github-actions]

Performance change found in the
test: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs for the metric: mean_load_model_latency_milli_secs.

For more information on how to triage the alerts, please look at
Triage performance alert issues section of the README.

Test description: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU.
Test link -

beam/.test-infra/jenkins/job_InferenceBenchmarkTests_Python.groovy

Line 151 in 42d0a6e

test : 'apache_beam.testing.benchmarks.inference.pytorch_image_classification_benchmarks',

Test dashboard - http://metrics.beam.apache.org/d/ZpS8Uf44z/python-ml-runinference-benchmarks?orgId=1&viewPanel=7

timestamp: Thu Feb 13 06:41:25 2025, metric_value: 119230.42
timestamp: Wed Feb 12 06:59:03 2025, metric_value: 114521.93
timestamp: Tue Feb 11 06:39:29 2025, metric_value: 118323.45
timestamp: Mon Feb 10 06:47:02 2025, metric_value: 118240.53
timestamp: Sun Feb  9 06:35:11 2025, metric_value: 117883.08 <---- Anomaly
timestamp: Sat Feb  8 06:56:17 2025, metric_value: 121331.05
timestamp: Fri Feb  7 06:27:45 2025, metric_value: 121596.68
timestamp: Thu Feb  6 06:28:36 2025, metric_value: 124385.60
timestamp: Wed Feb  5 06:38:04 2025, metric_value: 126043.03
timestamp: Tue Feb  4 06:22:29 2025, metric_value: 130307.28
timestamp: Mon Feb  3 06:14:52 2025, metric_value: 120352.85
timestamp: Sun Feb  2 06:25:12 2025, metric_value: 123105.87
timestamp: Sat Feb  1 06:19:08 2025, metric_value: 122917.00
timestamp: Fri Jan 31 06:15:15 2025, metric_value: 125189.10
timestamp: Wed Jan 29 06:34:31 2025, metric_value: 119985.23