Task
coreutils-static
GNU coreutils 9.7 is a collection of Linux utilities like ls, cp, mv, etc.
Project link: gnu.org/software/coreutils
Task: Compile and install all coreutils utilities to a specific location. Compile them statically.
Difficulties: Static linking requires correctly configuring the build.
Environment: Ubuntu 22.04 on amd64, without internet access.
Best attempt
| Model |
gpt-5-minimal
|
| Attempt |
0wf9zz975tfwg
|
| # of commands | 1 |
| Total time | 1m21s |
| Cost | $0.010 |
Terminal transcript >
compile-bench> set -e; export DEBIAN_FRONTEND=noninteractive; cd /home/peter; rm -rf build_coreutils; mkdir -p build_coreutils; tar -xf coreutils.tar.gz -C build_coreutils --strip-components=1; cd build_coreutils; echo "Configuring..."; ./configure --prefix=/home/peter/result --bindir=/home/peter/result/bin LDFLAGS='-static' CFLAGS='-O2 -pipe' CPPFLAGS='' --disable-nls; echo "Building..."; make -j"$(nproc)"; echo "Installing into staged prefix..."; make -j"$(nproc)" install; echo "Creating symlinks under /home/peter/result for each utility..."; mkdir -p /home/peter/result; for f in /home/peter/result/bin/*; do n=$(basename "$f"); ln -sf "bin/$n" "/home/peter/result/$n"; done; echo "Verifying static linkage..."; file /home/peter/result/bin/* | sed -n '1,5p'
Verifying static linkage...
/home/peter/result/bin/[: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=6bf2ba4b3ac37c40949f954e1ee93e82bb976a76, for GNU/Linux 3.2.0, not stripped
/home/peter/result/bin/b2sum: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=4b46b5d7015f37bea88d063042ddd235a9e5676a, for GNU/Linux 3.2.0, not stripped
/home/peter/result/bin/base32: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=ffcad848251369c89f39a429769655b9097fdb8a, for GNU/Linux 3.2.0, not stripped
/home/peter/result/bin/base64: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=efe0ff8f303c9696df46d6025b96a86b4af08f92, for GNU/Linux 3.2.0, not stripped
/home/peter/result/bin/basename: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=cd7edbd85957d0c6ed059ecd70eba34eda423c9a, for GNU/Linux 3.2.0, not stripped
Model ranking
| # | Model | ||||
|---|---|---|---|---|---|
| 1 |
gpt-oss-120b-high
|
100%
|
8 (8.0x) | 1m13s (1.1x) | $0.005 (2.3x) |
| 2 |
gemini-2.5-flash-thinking
|
100%
|
8 (8.0x) | 2m1s (1.7x) | $0.011 (5.3x) |
| 3 |
grok-4
|
100%
|
8 (8.0x) | 5m46s (5.0x) | $0.194 (89.0x) |
| 4 |
gemini-2.5-pro
|
100%
|
9 (9.0x) | 2m20s (2.0x) | $0.074 (34.2x) |
| 5 |
gpt-4.1
|
100%
|
10 (10.0x) | 1m21s (1.2x) | $0.059 (27.2x) |
| 6 |
grok-code-fast-1
|
100%
|
10 (10.0x) | 1m40s (1.4x) | $0.006 (3.0x) |
| 7 |
gpt-5-codex-high
|
100%
|
10 (10.0x) | 1m46s (1.5x) | $0.049 (22.5x) |
| 8 |
gpt-5-high
|
100%
|
10 (10.0x) | 3m41s (3.2x) | $0.117 (54.0x) |
| 9 |
gpt-5-mini-high
|
100%
|
11 (11.0x) | 3m10s (2.7x) | $0.014 (6.5x) |
| 10 |
claude-opus-4.1-thinking-16k
|
100%
|
14 (14.0x) | 3m18s (2.9x) | $0.619 (284.4x) |
| 11 |
claude-sonnet-4-thinking-16k
|
100%
|
16 (16.0x) | 2m39s (2.3x) | $0.161 (74.1x) |
| 12 |
deepseek-v3.1-terminus
|
100%
|
20 (20.0x) | 2m7s (1.8x) | $0.037 (17.1x) |
| 13 |
claude-sonnet-4.5-thinking-16k
|
100%
|
21 (21.0x) | 2m48s (2.4x) | $0.199 (91.3x) |
| 14 |
claude-sonnet-4
|
100%
|
24 (24.0x) | 2m21s (2.0x) | $0.185 (85.1x) |
| 15 |
claude-sonnet-4.5
|
100%
|
28 (28.0x) | 2m29s (2.1x) | $0.227 (104.1x) |
| 16 |
gpt-5-mini-minimal
|
67%
|
2 (2.0x) | 1m19s (1.1x) | $0.003 (1.3x) |
| 17 |
gpt-4.1-mini
|
67%
|
5 (5.0x) | 1m13s (1.1x) | $0.005 (2.3x) |
| 18 |
grok-4-fast
|
67%
|
7 (7.0x) | 1m59s (1.7x) | $0.002 (1.0x) |
| 19 |
glm-4.5
|
67%
|
8 (8.0x) | 1m33s (1.3x) | $0.021 (9.5x) |
| 20 |
gemini-2.5-flash
|
67%
|
9 (9.0x) | 1m44s (1.5x) | $0.009 (4.0x) |
| 21 |
deepseek-v3.1
|
67%
|
11 (11.0x) | 1m25s (1.2x) | $0.041 (18.9x) |
| 22 |
gpt-5-minimal
|
33%
|
1 (1.0x) | 1m21s (1.2x) | $0.010 (4.4x) |
| 23 |
qwen3-max
|
33%
|
3 (3.0x) | 1m10s (1.0x) | $0.008 (3.7x) |
| 24 |
kimi-k2-0905
|
33%
|
21 (21.0x) | 2m0s (1.7x) | $0.186 (85.5x) |
Attempts
| Model | Attempt ID | Cost | Duration | Status / Error |
|---|---|---|---|---|
|
claude-opus-4.1-thinking-16k
|
2a91dvdgom3h7
|
$0.560 | 3m6s | Success |
|
claude-opus-4.1-thinking-16k
|
90x3ffomvf2ec
|
$0.654 | 3m18s | Success |
|
claude-opus-4.1-thinking-16k
|
zvki8ok6bwoff
|
$0.619 | 3m25s | Success |
|
claude-sonnet-4
|
4um0qrearp9f9
|
$0.185 | 2m21s | Success |
|
claude-sonnet-4
|
kj9iicz6uy1ml
|
$0.182 | 2m11s | Success |
|
claude-sonnet-4
|
sd0y7c4jvrg8o
|
$0.192 | 2m23s | Success |
|
claude-sonnet-4-thinking-16k
|
0r24wkdxg86jb
|
$0.161 | 2m39s | Success |
|
claude-sonnet-4-thinking-16k
|
mrzt7ua84k80m
|
$0.166 | 2m21s | Success |
|
claude-sonnet-4-thinking-16k
|
plfws0uish2vi
|
$0.152 | 2m39s | Success |
|
claude-sonnet-4.5
|
5085o7137l1qc
|
$0.144 | 1m56s | Success |
|
claude-sonnet-4.5
|
pj3be723iad7p
|
$0.245 | 2m29s | Success |
|
claude-sonnet-4.5
|
xyr2pvg2ederi
|
$0.227 | 2m34s | Success |
|
claude-sonnet-4.5-thinking-16k
|
139n1l9ykqmay
|
$0.170 | 2m48s | Success |
|
claude-sonnet-4.5-thinking-16k
|
2cysws08wyagn
|
$0.199 | 2m35s | Success |
|
claude-sonnet-4.5-thinking-16k
|
a0fptc4dmrrb0
|
$0.199 | 2m48s | Success |
|
deepseek-v3.1
|
3fpu8zaowgq7t
|
$0.064 | 1m40s | Failure task failed: install missing at /home/peter/result/install or not executable |
|
deepseek-v3.1
|
53wz2rnf6ey8j
|
$0.041 | 1m25s | Success |
|
deepseek-v3.1
|
ro7f09l5pwvtd
|
$0.350 | 3m10s | Success |
|
deepseek-v3.1-terminus
|
6b4o82saybxkb
|
$0.042 | 2m28s | Success |
|
deepseek-v3.1-terminus
|
p9f3knux2bi0n
|
$0.032 | 1m48s | Success |
|
deepseek-v3.1-terminus
|
zbv4fu0bt59h5
|
$0.037 | 2m7s | Success |
|
gemini-2.5-flash
|
59gza3etqz54m
|
$0.028 | 4m40s | Success |
|
gemini-2.5-flash
|
mdp47e2isfz6l
|
$0.009 | 1m44s | Success |
|
gemini-2.5-flash
|
r3gc5tvqwi47d
|
$0.005 | 1m41s | Failure task failed: sha1sum binary does not exist |
|
gemini-2.5-flash-thinking
|
8894aqd63cabz
|
$0.014 | 1m57s | Success |
|
gemini-2.5-flash-thinking
|
cy7ku1xwju0xr
|
$0.009 | 2m7s | Success |
|
gemini-2.5-flash-thinking
|
lh8iwss0pb2sj
|
$0.011 | 2m1s | Success |
|
gemini-2.5-pro
|
i66fll9y9zbx8
|
$0.074 | 2m20s | Success |
|
gemini-2.5-pro
|
jm302v4jei7xk
|
$0.069 | 2m18s | Success |
|
gemini-2.5-pro
|
p5b244pg4blu1
|
$0.120 | 3m3s | Success |
|
glm-4.5
|
3efm44y4sq42v
|
$0.063 | 1m43s | Success |
|
glm-4.5
|
alraxhob6pahl
|
$0.021 | 1m33s | Success |
|
glm-4.5
|
eq6wgulo4zmts
|
$0.015 | 1m26s | Failure task failed: sha1sum binary does not exist |
|
gpt-4.1
|
8clcq0b9nvnp0
|
$0.059 | 1m19s | Success |
|
gpt-4.1
|
8yecdnfh30zng
|
$0.070 | 1m21s | Success |
|
gpt-4.1
|
my44jxbuog2ej
|
$0.050 | 2m7s | Success |
|
gpt-4.1-mini
|
i1gcm32w6y8lt
|
$0.020 | 1m28s | Failure task failed: install missing at /home/peter/result/install or not executable |
|
gpt-4.1-mini
|
m6jory3zzv0yy
|
$0.005 | 1m25s | Success |
|
gpt-4.1-mini
|
ny4qbms66bcj2
|
$0.005 | 1m13s | Success |
|
gpt-5-codex-high
|
bxnffu5scelp6
|
$0.041 | 1m46s | Success |
|
gpt-5-codex-high
|
dn62k0loubsof
|
$0.049 | 1m47s | Success |
|
gpt-5-codex-high
|
mpl6ujuxf9ozt
|
$0.055 | 1m42s | Success |
|
gpt-5-high
|
5lo26vu9b6whp
|
$0.093 | 3m22s | Success |
|
gpt-5-high
|
q3hyeaa3pax04
|
$0.121 | 3m58s | Success |
|
gpt-5-high
|
uqkifa3980qpy
|
$0.117 | 3m41s | Success |
|
gpt-5-mini-high
|
064sifofef33k
|
$0.020 | 3m10s | Success |
|
gpt-5-mini-high
|
asvvlggz0atlx
|
$0.014 | 3m12s | Success |
|
gpt-5-mini-high
|
dyhxix351i10v
|
$0.013 | 2m45s | Success |
|
gpt-5-mini-minimal
|
enorhmoh6t2fw
|
$0.003 | 1m19s | Success |
|
gpt-5-mini-minimal
|
lo37qtix0tzwi
|
$0.003 | 1m17s | Failure task failed: sha1sum is not statically linked |
|
gpt-5-mini-minimal
|
xstto1rh4k7wp
|
$0.004 | 1m22s | Success |
|
gpt-5-minimal
|
0wf9zz975tfwg
|
$0.010 | 1m21s | Success |
|
gpt-5-minimal
|
9fdga5w8t304a
|
$0.025 | 1m42s | Failure task failed: install missing at /home/peter/result/install or not executable |
|
gpt-5-minimal
|
dyjf0zqepyisi
|
$0.022 | 2m33s | Failure task failed: install missing at /home/peter/result/install or not executable |
|
gpt-oss-120b-high
|
nj2lwrvw8o3ce
|
$0.004 | 1m13s | Success |
|
gpt-oss-120b-high
|
oyyb2gl4c8ot8
|
$0.008 | 1m18s | Success |
|
gpt-oss-120b-high
|
rkeiuqn6aq5na
|
$0.005 | 1m13s | Success |
|
grok-4
|
nuou4bmvfyftj
|
$0.194 | 5m46s | Success |
|
grok-4
|
regzv7mzp9727
|
$0.165 | 3m39s | Success |
|
grok-4
|
z7ayimz2fks6a
|
$0.327 | 6m55s | Success |
|
grok-4-fast
|
9sop0p5sfu13x
|
$0.002 | 2m10s | Success |
|
grok-4-fast
|
j8h141f0u33a4
|
$0.005 | 1m59s | Success |
|
grok-4-fast
|
lierqs2hskl62
|
$0.004 | 2m1s | Failure task failed: sha1sum binary does not exist |
|
grok-code-fast-1
|
a23uog5dc0en8
|
$0.005 | 1m40s | Success |
|
grok-code-fast-1
|
g8z14y4lw1jn6
|
$0.009 | 2m13s | Success |
|
grok-code-fast-1
|
q4kta7ysm6iy1
|
$0.006 | 1m35s | Success |
|
kimi-k2-0905
|
6te6m7re2wqh9
|
$0.052 | 1m21s | Failure task failed: sha1sum binary does not exist |
|
kimi-k2-0905
|
rt01hrj1uetb0
|
$0.001 | 2s | Failure task failed: sha1sum binary does not exist |
|
kimi-k2-0905
|
w1wl60vs0iubq
|
$0.186 | 2m0s | Success |
|
qwen3-max
|
9ms8tpezozxkk
|
$0.028 | 1m3s | Failure task failed: sha1sum is not statically linked |
|
qwen3-max
|
g0lh92zoh2jah
|
$0.008 | 1m10s | Success |
|
qwen3-max
|
kispd61g3r4yo
|
$0.009 | 1m1s | Failure task failed: sha1sum is not statically linked |