Skip to content

Commit

Permalink
Merge pull request #71 from Xeygy/mlperf-inference-results-scc24
Browse files Browse the repository at this point in the history
Results on system scc102
  • Loading branch information
arjunsuresh authored Oct 28, 2024
2 parents 7abbcf5 + d5939d0 commit 65118c2
Show file tree
Hide file tree
Showing 22 changed files with 996 additions and 0 deletions.
1 change: 1 addition & 0 deletions open/scc102/code/bert-99/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
TBD
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
| Model | Scenario | Accuracy | Throughput | Latency (in ms) |
|---------|------------|------------|--------------|-------------------|
| bert-99 | offline | 88.7443 | 46.279 | - |
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
This experiment is generated using the [MLCommons Collective Mind automation framework (CM)](https://github.com/mlcommons/cm4mlops).

*Check [CM MLPerf docs](https://docs.mlcommons.org/inference) for more details.*

## Host platform

* OS version: Linux-6.1.110-1.el9.elrepo.x86_64-x86_64-with-glibc2.34
* CPU version: x86_64
* Python version: 3.9.18 (main, Aug 23 2024, 00:00:00)
[GCC 11.4.1 20231218 (Red Hat 11.4.1-3)]
* MLCommons CM version: 3.2.8

## CM Run Command

See [CM installation guide](https://docs.mlcommons.org/inference/install/).

```bash
pip install -U cmind

cm rm cache -f

cm pull repo mlcommons@cm4mlops --checkout=944c032d0381c97ab0fd0bbb622f1e53e63ab525

cm run script \
- \
- \
a \
d \
r \
. \
p \
y \
t \
h \
o \
n \
. \
n \
a \
m \
e \
= \
m \
l \
p \
e \
r \
f
```
*Note that if you want to use the [latest automation recipes](https://docs.mlcommons.org/inference) for MLPerf (CM scripts),
you should simply reload mlcommons@cm4mlops without checkout and clean CM cache as follows:*

```bash
cm rm repo mlcommons@cm4mlops
cm pull repo mlcommons@cm4mlops
cm rm cache -f

```

## Results

Platform: scc102_gpu0.novalocal-reference-gpu-pytorch-cu118

Model Precision: fp32

### Accuracy Results
`F1`: `88.74429`, Required accuracy for closed division `>= 89.96526`

### Performance Results
`Samples per second`: `46.2795`
Loading

0 comments on commit 65118c2

Please sign in to comment.