Task
curl
curl 8.16.0 is a command-line HTTP client. Project link: curl.se
Task: Compile and install curl to a specific location.
Difficulties: Standard build, nothing special.
Environment: Ubuntu 22.04 on amd64, without internet access.
Best attempt
| Model |
gpt-5-minimal
|
| Attempt |
b5bj7y1ehlylx
|
| # of commands | 1 |
| Total time | 30s |
| Cost | $0.009 |
Terminal transcript >
compile-bench> set -e; export DEBIAN_FRONTEND=noninteractive; rm -rf build_curl && mkdir build_curl && tar -xf /home/peter/curl.tar.gz -C build_curl && cd build_curl/* && printf 'Configuring...
' && ./configure --prefix=/home/peter/result --disable-ldap --disable-ldaps --disable-rtsp --disable-dict --disable-telnet --disable-tftp --disable-pop3 --disable-imap --disable-smtp --disable-rtmp --disable-gopher --disable-smb --disable-ftp --disable-libcurl-option --without-ssl --without-gnutls --without-mbedtls --without-wolfssl --without-bearssl --without-nss --without-libpsl --without-libidn2 --without-librtmp --without-brotli --without-zstd --without-zlib --without-nghttp2 --without-nghttp3 --without-ngtcp2 --without-libssh2 --without-libssh --without-libgsasl --without-libuv --without-rtmpdump --disable-shared --enable-static && printf 'Building...
' && make -j$(nproc) V=1 && printf 'Installing...
' && make install && ln -sfn /home/peter/result/bin/curl /home/peter/result/curl && /home/peter/result/bin/curl --version
make[2]: Leaving directory '/home/peter/build_curl/curl-8.16.0'
make[1]: Leaving directory '/home/peter/build_curl/curl-8.16.0'
curl 8.16.0 (x86_64-pc-linux-gnu) libcurl/8.16.0
Release-Date: 2025-09-10
Protocols: file http ipfs ipns mqtt ws
Features: alt-svc AsynchDNS IPv6 Largefile threadsafe UnixSockets
Model ranking
| # | Model | ||||
|---|---|---|---|---|---|
| 1 |
gpt-5-minimal
|
100%
|
1 (1.0x) | 40s (1.4x) | $0.010 (2.4x) |
| 2 |
qwen3-max
|
100%
|
3 (3.0x) | 29s (1.0x) | $0.008 (2.1x) |
| 3 |
gpt-5-high
|
100%
|
4 (4.0x) | 2m7s (4.4x) | $0.059 (15.0x) |
| 4 |
gpt-4.1
|
100%
|
8 (8.0x) | 31s (1.1x) | $0.034 (8.8x) |
| 5 |
gemini-2.5-pro
|
100%
|
8 (8.0x) | 2m9s (4.5x) | $0.057 (14.5x) |
| 6 |
gpt-5-codex-high
|
100%
|
9 (9.0x) | 1m2s (2.1x) | $0.040 (10.2x) |
| 7 |
claude-opus-4.1-thinking-16k
|
100%
|
9 (9.0x) | 1m51s (3.8x) | $0.443 (112.8x) |
| 8 |
grok-4
|
100%
|
9 (9.0x) | 2m55s (6.0x) | $0.120 (30.5x) |
| 9 |
deepseek-v3.1
|
100%
|
10 (10.0x) | 49s (1.7x) | $0.033 (8.4x) |
| 10 |
gpt-4.1-mini
|
100%
|
11 (11.0x) | 42s (1.4x) | $0.012 (3.0x) |
| 11 |
gpt-5-mini-high
|
100%
|
12 (12.0x) | 2m19s (4.8x) | $0.015 (3.8x) |
| 12 |
claude-sonnet-4.5-thinking-16k
|
100%
|
13 (13.0x) | 1m29s (3.1x) | $0.122 (31.0x) |
| 13 |
claude-sonnet-4-thinking-16k
|
100%
|
13 (13.0x) | 1m36s (3.3x) | $0.129 (32.9x) |
| 14 |
deepseek-v3.1-terminus
|
100%
|
14 (14.0x) | 1m17s (2.7x) | $0.026 (6.6x) |
| 15 |
claude-sonnet-4.5
|
100%
|
15 (15.0x) | 1m7s (2.3x) | $0.110 (28.0x) |
| 16 |
kimi-k2-0905
|
100%
|
17 (17.0x) | 1m2s (2.1x) | $0.168 (42.8x) |
| 17 |
claude-sonnet-4
|
100%
|
18 (18.0x) | 1m26s (3.0x) | $0.207 (52.8x) |
| 18 |
gemini-2.5-flash
|
67%
|
5 (5.0x) | 29s (1.0x) | $0.004 (1.0x) |
| 19 |
gemini-2.5-flash-thinking
|
67%
|
5 (5.0x) | 32s (1.1x) | $0.005 (1.3x) |
| 20 |
grok-code-fast-1
|
67%
|
10 (10.0x) | 1m40s (3.4x) | $0.006 (1.5x) |
| 21 |
grok-4-fast
|
33%
|
8 (8.0x) | 1m11s (2.4x) | $0.004 (1.1x) |
| 22 |
glm-4.5
|
33%
|
12 (12.0x) | 2m16s (4.7x) | $0.038 (9.7x) |
| 23 |
gpt-oss-120b-high
|
33%
|
14 (14.0x) | 49s (1.7x) | $0.013 (3.4x) |
| 24 |
gpt-5-mini-minimal
|
0%
|
Attempts
| Model | Attempt ID | Cost | Duration | Status / Error |
|---|---|---|---|---|
|
claude-opus-4.1-thinking-16k
|
dfv4pts3rxcp8
|
$0.443 | 1m51s | Success |
|
claude-opus-4.1-thinking-16k
|
fzj1tcx9x3qe2
|
$0.413 | 1m34s | Success |
|
claude-opus-4.1-thinking-16k
|
vplzlg523e6yt
|
$0.564 | 2m12s | Success |
|
claude-sonnet-4
|
31m180yp96sg4
|
$0.207 | 1m38s | Success |
|
claude-sonnet-4
|
jzcym615o3bb1
|
$0.120 | 1m22s | Success |
|
claude-sonnet-4
|
mlaw9i67o9ko8
|
$0.211 | 1m26s | Success |
|
claude-sonnet-4-thinking-16k
|
3u30iilci3bu2
|
$0.129 | 1m36s | Success |
|
claude-sonnet-4-thinking-16k
|
b8j5yhsiyefwy
|
$0.126 | 1m40s | Success |
|
claude-sonnet-4-thinking-16k
|
mhjy1qvhvzpzd
|
$0.157 | 1m25s | Success |
|
claude-sonnet-4.5
|
a82s2nkx6wvzs
|
$0.099 | 1m4s | Success |
|
claude-sonnet-4.5
|
c783hcdqqy6aq
|
$0.110 | 1m7s | Success |
|
claude-sonnet-4.5
|
cp1f686cylfhk
|
$0.120 | 1m13s | Success |
|
claude-sonnet-4.5-thinking-16k
|
c4vu7pxfviz4x
|
$0.115 | 1m24s | Success |
|
claude-sonnet-4.5-thinking-16k
|
escystgb99u4j
|
$0.160 | 1m56s | Success |
|
claude-sonnet-4.5-thinking-16k
|
zfjv1m8ipehsr
|
$0.122 | 1m29s | Success |
|
deepseek-v3.1
|
1vs1dnj1rbh0h
|
$0.025 | 53s | Success |
|
deepseek-v3.1
|
azdd243riac1n
|
$0.033 | 49s | Success |
|
deepseek-v3.1
|
uip9ful0s1c7e
|
$0.033 | 49s | Success |
|
deepseek-v3.1-terminus
|
66vdrw1zk49zc
|
$0.026 | 1m17s | Success |
|
deepseek-v3.1-terminus
|
du2902p38frkz
|
$0.023 | 1m11s | Success |
|
deepseek-v3.1-terminus
|
sb9e19g4y17s9
|
$0.039 | 1m47s | Success |
|
gemini-2.5-flash
|
27bz01qbbllnb
|
$0.004 | 29s | Success |
|
gemini-2.5-flash
|
jelgpn94ryjy7
|
$0.010 | 32s | Failure task failed: curl did not download the expected local file content, but instead: curl: (1) Protocol "file" not supported |
|
gemini-2.5-flash
|
l0al12nsj8980
|
$0.023 | 48s | Success |
|
gemini-2.5-flash-thinking
|
4mbumgeh26inc
|
$0.024 | 1m16s | Success |
|
gemini-2.5-flash-thinking
|
ecp5uv5wyqr2r
|
$0.006 | 35s | Failure task failed: curl did not download the expected local file content, but instead: curl: (1) Protocol "file" not supported |
|
gemini-2.5-flash-thinking
|
vw1lcqlbfzuhu
|
$0.005 | 32s | Success |
|
gemini-2.5-pro
|
3e4jro69dm5ox
|
$0.055 | 2m6s | Success |
|
gemini-2.5-pro
|
b3x2q9982j05r
|
$0.057 | 2m9s | Success |
|
gemini-2.5-pro
|
dbyhg0dlp3joc
|
$0.279 | 4m27s | Success |
|
glm-4.5
|
76i3akdbzlr3k
|
$0.038 | 2m16s | Success |
|
glm-4.5
|
fppvhgc27jtmo
|
$0.025 | 50s | Failure task failed: curl binary does not exist |
|
glm-4.5
|
we96hpilygjsr
|
$0.031 | 1m58s | Failure task failed: curl binary does not exist |
|
gpt-4.1
|
3xns9ceaom0ul
|
$0.037 | 31s | Success |
|
gpt-4.1
|
bwxa5zt72aypx
|
$0.030 | 30s | Success |
|
gpt-4.1
|
x8ypw6yyfrflf
|
$0.034 | 33s | Success |
|
gpt-4.1-mini
|
832y21jw8tb5e
|
$0.002 | 23s | Success |
|
gpt-4.1-mini
|
aewcboi7hghxu
|
$0.034 | 1m17s | Success |
|
gpt-4.1-mini
|
ofs50jzdw2i0b
|
$0.012 | 42s | Success |
|
gpt-5-codex-high
|
fvxyoeqvih89p
|
$0.053 | 1m5s | Success |
|
gpt-5-codex-high
|
jte90pyegwbsw
|
$0.040 | 1m2s | Success |
|
gpt-5-codex-high
|
w5l89it2ratfa
|
$0.036 | 46s | Success |
|
gpt-5-high
|
1kpgqsl29yngn
|
$0.059 | 2m7s | Success |
|
gpt-5-high
|
5j52ohabvhzs4
|
$0.055 | 1m57s | Success |
|
gpt-5-high
|
kg4nv8jv9trqa
|
$0.061 | 2m11s | Success |
|
gpt-5-mini-high
|
9x5acpazx08ff
|
$0.046 | 5m5s | Success |
|
gpt-5-mini-high
|
ah443gb5vivip
|
$0.015 | 2m19s | Success |
|
gpt-5-mini-high
|
ofev8fggvoz6a
|
$0.013 | 2m17s | Success |
|
gpt-5-mini-minimal
|
5xm729eayko85
|
$0.004 | 1m0s | Failure task failed: curl did not download the expected local file content, but instead: curl: (1) Protocol "file" not supported |
|
gpt-5-mini-minimal
|
drc17c5616vr4
|
$0.010 | 1m7s | Failure task failed: curl binary does not exist |
|
gpt-5-mini-minimal
|
yxo4dtf5738u7
|
$0.011 | 59s | Failure task failed: curl binary does not exist |
|
gpt-5-minimal
|
b5bj7y1ehlylx
|
$0.009 | 30s | Success |
|
gpt-5-minimal
|
s0qj05bfccji4
|
$0.010 | 40s | Success |
|
gpt-5-minimal
|
wmzrhk9cefavg
|
$0.018 | 47s | Success |
|
gpt-oss-120b-high
|
1x4wr3w05gfq3
|
$0.008 | 55s | Failure task failed: curl did not download the expected local file content, but instead: curl: (1) Protocol "file" not supported |
|
gpt-oss-120b-high
|
70edkrxd6zgxu
|
$0.013 | 49s | Success |
|
gpt-oss-120b-high
|
wen54d55zmlxe
|
$0.000 | 21s | Failure task failed: curl binary does not exist |
|
grok-4
|
96qnzr1qyss7v
|
$0.120 | 1m52s | Success |
|
grok-4
|
hhxw07fc7lxkl
|
$0.073 | 2m55s | Success |
|
grok-4
|
i6s755av5byv0
|
$0.162 | 3m32s | Success |
|
grok-4-fast
|
cs0mhhqdeag8y
|
$0.002 | 1m31s | Failure task failed: curl did not download the expected local file content, but instead: curl: (1) Protocol "file" not supported |
|
grok-4-fast
|
mjrzsu8rx73gb
|
$0.004 | 2m21s | Failure task failed: curl binary does not exist |
|
grok-4-fast
|
tbwh2roaybvlb
|
$0.004 | 1m11s | Success |
|
grok-code-fast-1
|
0plz63ws5i2r0
|
$0.007 | 1m53s | Success |
|
grok-code-fast-1
|
q6m0dvkupep5t
|
$0.004 | 1m47s | Failure task failed: curl did not download the expected local file content, but instead: curl: (1) Protocol "file" not supported |
|
grok-code-fast-1
|
zliwyxsva0197
|
$0.006 | 1m40s | Success |
|
kimi-k2-0905
|
4vyy5x2zkq95p
|
$0.085 | 44s | Success |
|
kimi-k2-0905
|
a95z2kutwa5v5
|
$0.168 | 1m30s | Success |
|
kimi-k2-0905
|
odufbvquul9dq
|
$0.174 | 1m2s | Success |
|
qwen3-max
|
i2qbyf40pfcj0
|
$0.008 | 28s | Success |
|
qwen3-max
|
urvuyc2twiz3t
|
$0.008 | 29s | Success |
|
qwen3-max
|
xx1yp395zaha1
|
$0.008 | 30s | Success |