Model
grok-4
Performance of grok-4 across tasks.
Task ranking
| # | Task | ||||
|---|---|---|---|---|---|
| 1 | coreutils |
100%
|
6 (2.0x) | 2m58s (2.1x) | $0.094 (1.0x) |
| 2 | cowsay |
100%
|
8 (2.7x) | 1m26s (1.0x) | $0.112 (1.2x) |
| 3 | jq |
100%
|
8 (2.7x) | 1m50s (1.3x) | $0.110 (1.2x) |
| 4 | coreutils-static |
100%
|
8 (2.7x) | 5m46s (4.0x) | $0.194 (2.1x) |
| 5 | curl |
100%
|
9 (3.0x) | 2m55s (2.0x) | $0.120 (1.3x) |
| 6 | curl-ssl |
100%
|
9 (3.0x) | 3m6s (2.2x) | $0.160 (1.7x) |
| 7 | coreutils-old-version |
100%
|
12 (4.0x) | 5m47s (4.0x) | $0.243 (2.6x) |
| 8 | jq-static |
100%
|
16 (5.3x) | 4m24s (3.1x) | $0.374 (4.0x) |
| 9 | jq-windows |
67%
|
3 (1.0x) | 3m5s (2.1x) | $0.155 (1.6x) |
| 10 | jq-windows2 |
67%
|
4 (1.3x) | 2m43s (1.9x) | $0.108 (1.1x) |
| 11 | coreutils-static-alpine |
67%
|
9 (3.0x) | 6m16s (4.4x) | $0.247 (2.6x) |
| 12 | jq-static-musl |
33%
|
10 (3.3x) | 2m28s (1.7x) | $0.147 (1.6x) |
| 13 | curl-ssl-arm64-static2 |
33%
|
29 (9.7x) | 10m29s (7.3x) | $1.041 (11.0x) |
| 14 | coreutils-old-version-alpine |
0%
|
|||
| 15 | curl-ssl-arm64-static |
0%
|
Attempts
| Task | Attempt ID | Cost | Duration | Status / Error |
|---|---|---|---|---|
| coreutils |
4l6wc81hrglzu
|
$0.094 | 3m13s | Success |
| coreutils |
bt7d1qa49srbk
|
$0.098 | 2m56s | Success |
| coreutils |
ktl5tvskto1tt
|
$0.077 | 2m58s | Success |
| coreutils-old-version |
5p3vlguat3wv5
|
$0.203 | 4m46s | Success |
| coreutils-old-version |
f42wz7er54s06
|
$0.243 | 7m27s | Success |
| coreutils-old-version |
ywe12bzjsdojn
|
$0.298 | 5m47s | Success |
| coreutils-old-version-alpine |
0v7qjmu94o0w5
|
$1.836 | 23m29s | Failure task failed: df missing at /home/peter/result/df or not executable |
| coreutils-old-version-alpine |
ffj48x6sy4918
|
$7.081 | 24m5s | Failure task failed: sha1sum binary does not exist |
| coreutils-old-version-alpine |
ob89w3v71ur3b
|
$3.753 | 17m24s | Failure task failed: df missing at /home/peter/result/df or not executable |
| coreutils-static |
nuou4bmvfyftj
|
$0.194 | 5m46s | Success |
| coreutils-static |
regzv7mzp9727
|
$0.165 | 3m39s | Success |
| coreutils-static |
z7ayimz2fks6a
|
$0.327 | 6m55s | Success |
| coreutils-static-alpine |
d73avs395xe5y
|
$0.247 | 6m16s | Success |
| coreutils-static-alpine |
s7aguiggk23s5
|
$0.752 | 9m2s | Success |
| coreutils-static-alpine |
yzdb8yjou3nn8
|
$0.254 | 14m57s | Failure context timeout: context deadline exceeded |
| cowsay |
djw9wg2h6exct
|
$0.281 | 4m41s | Success |
| cowsay |
go3byi2cajhq9
|
$0.075 | 1m21s | Success |
| cowsay |
wqoodmrn3qfu8
|
$0.112 | 1m26s | Success |
| curl |
96qnzr1qyss7v
|
$0.120 | 1m52s | Success |
| curl |
hhxw07fc7lxkl
|
$0.073 | 2m55s | Success |
| curl |
i6s755av5byv0
|
$0.162 | 3m32s | Success |
| curl-ssl |
5cag5qqdwln1h
|
$0.133 | 2m2s | Success |
| curl-ssl |
5x88xjnryhhs2
|
$0.177 | 3m6s | Success |
| curl-ssl |
d44ff6zw05z04
|
$0.160 | 3m11s | Success |
| curl-ssl-arm64-static |
23avhe5la4cpz
|
$1.725 | 19m51s | Failure task failed: curl-arm64 is not statically linked |
| curl-ssl-arm64-static |
am3bi8pw34usd
|
$1.304 | 12m53s | Failure task failed: curl HTTPS request to google.com did not return content-type: text/html but instead: } [2 bytes data] * SSL... |
| curl-ssl-arm64-static |
tqqqhissqfqyi
|
$1.129 | 17m26s | Failure task failed: curl-arm64 is not statically linked |
| curl-ssl-arm64-static2 |
pzj91ruap1va2
|
$0.424 | 6m31s | Failure task failed: curl-arm64 is not statically linked |
| curl-ssl-arm64-static2 |
qcleqkmmkoxqq
|
$0.566 | 5m27s | Failure task failed: curl binary does not exist |
| curl-ssl-arm64-static2 |
whc01txfz0nhf
|
$1.041 | 10m29s | Success |
| jq |
1fbpcmjw7hshm
|
$0.120 | 1m50s | Success |
| jq |
2et9jg34j40ag
|
$0.092 | 1m36s | Success |
| jq |
wufdaiz3qjkbo
|
$0.110 | 2m3s | Success |
| jq-static |
rhxdxbjo1uunc
|
$0.212 | 2m20s | Success |
| jq-static |
t34l88gctqmu4
|
$0.713 | 11m11s | Success |
| jq-static |
wnadxk7l3zi7j
|
$0.374 | 4m24s | Success |
| jq-static-musl |
izbh668vbws1p
|
$0.268 | 3m45s | Failure task failed: jq binary does not exist |
| jq-static-musl |
k0as8rwlez788
|
$0.147 | 2m28s | Success |
| jq-static-musl |
n6160x2rlj02d
|
$0.116 | 2m8s | Failure task failed: jq binary does not exist |
| jq-windows |
7xywc270dfxvt
|
$0.155 | 3m23s | Success |
| jq-windows |
jhiphmji70f2h
|
$0.105 | 2m5s | Failure task failed: jq help does not contain expected string |
| jq-windows |
wakd5ievn14m3
|
$0.164 | 3m5s | Success |
| jq-windows2 |
bedwzggytaupw
|
$0.313 | 5m58s | Success |
| jq-windows2 |
fdnj8zihhkeg2
|
$2.146 | 15m26s | Failure task failed: jq help does not contain expected string |
| jq-windows2 |
r50ycqek99hix
|
$0.108 | 2m43s | Success |