Task
coreutils

GNU coreutils 9.7 is a collection of Linux utilities like ls, cp, mv, etc. Project link: gnu.org/software/coreutils

Task: Compile and install all coreutils utilities to a specific location.

Difficulties: Standard autotools setup, nothing special.

Environment: Ubuntu 22.04 on amd64, without internet access.

Best attempt
Model gpt-5-mini-high logo gpt-5-mini-high
Attempt 71rfh3egedqj4
# of commands 2
Total time 3m5s
Cost $0.014
Terminal transcript >
compile-bench> set -e; rm -rf /home/peter/result; mkdir -p /home/peter/result; tar -xzf coreutils.tar.gz; srcdir=$(tar -tzf coreutils.tar.gz | head -1 | cut -f1 -d"/"); cd "$srcdir"; if ! ./configure --prefix=/home/peter/result; then echo "configure failed, running autoreconf and retrying"; autoreconf -fi; ./configure --prefix=/home/peter/result; fi; if ! make -j$(nproc); then echo "make failed, retrying with -j1"; make -j1; fi; make install; for f in /home/peter/result/bin/*; do if [ -f "$f" -a -x "$f" ]; then ln -sf "$f" "/home/peter/result/$(basename "$f")"; fi; done; echo "Symlink sample:"; ls -l /home/peter/result/uptime || true; /home/peter/result/uptime --version 2>/dev/null || true; echo "Done"
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Joseph Arceneaux, David MacKenzie, and Kaveh Ghazi.
Done
compile-bench> echo 'exes in bin:'; find /home/peter/result/bin -maxdepth 1 -type f -executable -printf '%f
' | sort | sed -n '1,200p'; echo 'count in bin:'; find /home/peter/result/bin -maxdepth 1 -type f -executable | wc -l; echo 'symlinks in /home/peter/result:'; find /home/peter/result -maxdepth 1 -type l -printf '%f -> %l
' | sed -n '1,200p'; echo 'count symlinks:'; find /home/peter/result -maxdepth 1 -type l | wc -l
id -> /home/peter/result/bin/id
join -> /home/peter/result/bin/join
env -> /home/peter/result/bin/env
fmt -> /home/peter/result/bin/fmt
count symlinks:
106
Model ranking
# Model
1
gpt-5-mini-high logo gpt-5-mini-high
100%
2 (1.0x) 3m15s (3.4x) $0.016 (5.9x)
2
gpt-4.1-mini logo gpt-4.1-mini
100%
5 (2.5x) 57s (1.0x) $0.007 (2.4x)
3
qwen3-max logo qwen3-max
100%
5 (2.5x) 1m3s (1.1x) $0.022 (8.1x)
4
gpt-5-high logo gpt-5-high
100%
6 (3.0x) 2m33s (2.7x) $0.064 (23.3x)
5
grok-4 logo grok-4
100%
6 (3.0x) 2m58s (3.1x) $0.094 (34.6x)
6
gemini-2.5-flash-thinking logo gemini-2.5-flash-thinking
100%
8 (4.0x) 1m9s (1.2x) $0.011 (3.9x)
7
gemini-2.5-pro logo gemini-2.5-pro
100%
8 (4.0x) 1m48s (1.9x) $0.049 (18.0x)
8
gpt-oss-120b-high logo gpt-oss-120b-high
100%
9 (4.5x) 1m2s (1.1x) $0.006 (2.1x)
9
grok-code-fast-1 logo grok-code-fast-1
100%
9 (4.5x) 1m36s (1.7x) $0.004 (1.6x)
10
gpt-5-codex-high logo gpt-5-codex-high
100%
10 (5.0x) 1m33s (1.6x) $0.047 (17.4x)
11
gpt-4.1 logo gpt-4.1
100%
11 (5.5x) 1m7s (1.2x) $0.051 (18.9x)
12
claude-opus-4.1-thinking-16k logo claude-opus-4.1-thinking-16k
100%
12 (6.0x) 2m50s (3.0x) $0.566 (208.1x)
13
deepseek-v3.1 logo deepseek-v3.1
100%
13 (6.5x) 1m23s (1.5x) $0.041 (15.1x)
14
glm-4.5 logo glm-4.5
100%
14 (7.0x) 1m35s (1.7x) $0.041 (15.2x)
15
claude-sonnet-4-thinking-16k logo claude-sonnet-4-thinking-16k
100%
16 (8.0x) 2m12s (2.3x) $0.149 (54.9x)
16
deepseek-v3.1-terminus logo deepseek-v3.1-terminus
100%
19 (9.5x) 1m56s (2.0x) $0.037 (13.6x)
17
claude-sonnet-4 logo claude-sonnet-4
100%
20 (10.0x) 1m57s (2.1x) $0.153 (56.3x)
18
claude-sonnet-4.5-thinking-16k logo claude-sonnet-4.5-thinking-16k
100%
20 (10.0x) 2m24s (2.5x) $0.168 (61.9x)
19
claude-sonnet-4.5 logo claude-sonnet-4.5
100%
25 (12.5x) 2m2s (2.1x) $0.179 (65.9x)
20
gpt-5-minimal logo gpt-5-minimal
67%
4 (2.0x) 1m9s (1.2x) $0.014 (5.2x)
21
grok-4-fast logo grok-4-fast
67%
7 (3.5x) 1m22s (1.5x) $0.003 (1.2x)
22
gemini-2.5-flash logo gemini-2.5-flash
67%
8 (4.0x) 1m26s (1.5x) $0.007 (2.5x)
23
kimi-k2-0905 logo kimi-k2-0905
67%
15 (7.5x) 1m20s (1.4x) $0.124 (45.6x)
24
gpt-5-mini-minimal logo gpt-5-mini-minimal
33%
3 (1.5x) 1m3s (1.1x) $0.003 (1.0x)
Attempts
Model Attempt ID Cost Duration Status / Error
claude-opus-4.1-thinking-16k logo claude-opus-4.1-thinking-16k
e28iidf9k7lni $0.566 2m55s Success
claude-opus-4.1-thinking-16k logo claude-opus-4.1-thinking-16k
fq8lve965t7ka $0.494 2m23s Success
claude-opus-4.1-thinking-16k logo claude-opus-4.1-thinking-16k
t5kmlo7zth3u9 $0.603 2m50s Success
claude-sonnet-4 logo claude-sonnet-4
5yetcksbd1rpe $0.135 1m57s Success
claude-sonnet-4 logo claude-sonnet-4
731fv5dv8u8xl $0.153 1m56s Success
claude-sonnet-4 logo claude-sonnet-4
v1kvc3t6y2j41 $0.163 2m3s Success
claude-sonnet-4-thinking-16k logo claude-sonnet-4-thinking-16k
68zatcuqtx2wy $0.142 2m8s Success
claude-sonnet-4-thinking-16k logo claude-sonnet-4-thinking-16k
slpk3bbep7scl $0.149 2m28s Success
claude-sonnet-4-thinking-16k logo claude-sonnet-4-thinking-16k
ykldbm7wfb1e0 $0.152 2m12s Success
claude-sonnet-4.5 logo claude-sonnet-4.5
9qc9g6eal3n8n $0.168 2m2s Success
claude-sonnet-4.5 logo claude-sonnet-4.5
gduoax5ti0zz0 $0.179 2m3s Success
claude-sonnet-4.5 logo claude-sonnet-4.5
nkfbsgibraefv $0.208 1m58s Success
claude-sonnet-4.5-thinking-16k logo claude-sonnet-4.5-thinking-16k
s4uqbdubq5p0n $0.168 2m24s Success
claude-sonnet-4.5-thinking-16k logo claude-sonnet-4.5-thinking-16k
sy4n89mgtd7q0 $0.196 3m17s Success
claude-sonnet-4.5-thinking-16k logo claude-sonnet-4.5-thinking-16k
zk25bwmdcel73 $0.130 2m1s Success
deepseek-v3.1 logo deepseek-v3.1
2s2z7s4b8azq2 $0.026 1m5s Success
deepseek-v3.1 logo deepseek-v3.1
l6rk5xweqdqj9 $0.057 1m23s Success
deepseek-v3.1 logo deepseek-v3.1
ua1anvszt97g4 $0.041 1m26s Success
deepseek-v3.1-terminus logo deepseek-v3.1-terminus
iv1bobpame97y $0.037 1m56s Success
deepseek-v3.1-terminus logo deepseek-v3.1-terminus
myzu59fv706ck $0.037 1m47s Success
deepseek-v3.1-terminus logo deepseek-v3.1-terminus
n313dy6qcm216 $0.054 2m27s Success
gemini-2.5-flash logo gemini-2.5-flash
72mffpubnnu4n $0.007 1m26s Success
gemini-2.5-flash logo gemini-2.5-flash
cizcdb805itzg $0.017 1m37s Failure task failed: sha1sum binary does not exist
gemini-2.5-flash logo gemini-2.5-flash
l5900rmi59rl4 $0.007 1m27s Success
gemini-2.5-flash-thinking logo gemini-2.5-flash-thinking
0jjoczrhw7t0d $0.011 1m10s Success
gemini-2.5-flash-thinking logo gemini-2.5-flash-thinking
3vzn142dc7wzu $0.011 1m8s Success
gemini-2.5-flash-thinking logo gemini-2.5-flash-thinking
9ughwpxv3ykiu $0.010 1m9s Success
gemini-2.5-pro logo gemini-2.5-pro
mavjj3haoinsc $0.055 1m47s Success
gemini-2.5-pro logo gemini-2.5-pro
n2xiawao4lzf7 $0.042 1m48s Success
gemini-2.5-pro logo gemini-2.5-pro
nkt2id5mi09oq $0.049 1m53s Success
glm-4.5 logo glm-4.5
2hwexo41qktac $0.101 2m30s Success
glm-4.5 logo glm-4.5
82872ya0h76zh $0.041 1m27s Success
glm-4.5 logo glm-4.5
h85c5q0g7mqw4 $0.038 1m35s Success
gpt-4.1 logo gpt-4.1
8m33hbo8fhu4v $0.055 1m6s Success
gpt-4.1 logo gpt-4.1
j5v8pmha07eo7 $0.051 1m12s Success
gpt-4.1 logo gpt-4.1
s45jh9t5t8cb8 $0.036 1m7s Success
gpt-4.1-mini logo gpt-4.1-mini
2p70j91hufzfc $0.003 56s Success
gpt-4.1-mini logo gpt-4.1-mini
g0ufozl8ppx4u $0.007 1m8s Success
gpt-4.1-mini logo gpt-4.1-mini
hkho8smf3400h $0.007 57s Success
gpt-5-codex-high logo gpt-5-codex-high
elm94zto0xjiy $0.042 1m31s Success
gpt-5-codex-high logo gpt-5-codex-high
jya2quzbkbt46 $0.047 1m33s Success
gpt-5-codex-high logo gpt-5-codex-high
vds73s9n1nh88 $0.053 1m41s Success
gpt-5-high logo gpt-5-high
59x5dbjrvlewq $0.064 2m33s Success
gpt-5-high logo gpt-5-high
6bygfzztwt9gk $0.061 2m32s Success
gpt-5-high logo gpt-5-high
x4ia5x2kascfe $0.082 3m32s Success
gpt-5-mini-high logo gpt-5-mini-high
71rfh3egedqj4 $0.014 3m5s Success
gpt-5-mini-high logo gpt-5-mini-high
9roqytc4zztxx $0.016 3m15s Success
gpt-5-mini-high logo gpt-5-mini-high
ozg4c7ll3hitx $0.019 3m29s Success
gpt-5-mini-minimal logo gpt-5-mini-minimal
8kdbqlawdlfsk $0.003 1m10s Failure task failed: sha1sum binary does not exist
gpt-5-mini-minimal logo gpt-5-mini-minimal
j916wqm4dkq54 $0.003 1m9s Failure task failed: sha1sum binary does not exist
gpt-5-mini-minimal logo gpt-5-mini-minimal
xlwu5dg5wooqr $0.003 1m3s Success
gpt-5-minimal logo gpt-5-minimal
2340iu0u91fy4 $0.041 1m38s Failure task failed: cat missing at /home/peter/result/cat or not executable
gpt-5-minimal logo gpt-5-minimal
2ugptzs6vmzu6 $0.042 2m25s Success
gpt-5-minimal logo gpt-5-minimal
zfx0vqwf0t10l $0.014 1m9s Success
gpt-oss-120b-high logo gpt-oss-120b-high
g6sqx3qa2dd5a $0.004 1m6s Success
gpt-oss-120b-high logo gpt-oss-120b-high
j47ieqotalzlp $0.006 1m1s Success
gpt-oss-120b-high logo gpt-oss-120b-high
nfb82u432ez9c $0.007 1m2s Success
grok-4 logo grok-4
4l6wc81hrglzu $0.094 3m13s Success
grok-4 logo grok-4
bt7d1qa49srbk $0.098 2m56s Success
grok-4 logo grok-4
ktl5tvskto1tt $0.077 2m58s Success
grok-4-fast logo grok-4-fast
figh60llewnbw $0.002 1m56s Failure task failed: sha1sum binary does not exist
grok-4-fast logo grok-4-fast
n98fpmzucrnzi $0.003 1m22s Success
grok-4-fast logo grok-4-fast
xvu775toqwjk1 $0.004 1m48s Success
grok-code-fast-1 logo grok-code-fast-1
0us03vpgc9in7 $0.004 1m35s Success
grok-code-fast-1 logo grok-code-fast-1
6lh05jz58mbf5 $0.004 1m36s Success
grok-code-fast-1 logo grok-code-fast-1
h5nur77w8jk47 $0.004 1m44s Success
kimi-k2-0905 logo kimi-k2-0905
gargqot0mggui $0.336 2m23s Success
kimi-k2-0905 logo kimi-k2-0905
kexsuc8y1btty $0.124 1m20s Success
kimi-k2-0905 logo kimi-k2-0905
ldv01taynuu62 $0.110 1m52s Failure task failed: sha1sum binary does not exist
qwen3-max logo qwen3-max
bzy6w8v80p21w $0.021 1m0s Success
qwen3-max logo qwen3-max
j6w5331r9yzlb $0.022 1m4s Success
qwen3-max logo qwen3-max
tvww9mo78fnw3 $0.028 1m3s Success