Task
jq-static
jq 1.8.1 is a command-line JSON utility for viewing and transforming JSON. Project link: github.com/jqlang/jq
Task: Compile and install statically-linked jq to a specific location.
Difficulties: Static linking requires correctly configuring the build.
Environment: Ubuntu 22.04 on amd64, without internet access.
Best attempt
| Model |
gpt-5-minimal
|
| Attempt |
r6mzoz6jd0dcb
|
| # of commands | 2 |
| Total time | 37s |
| Cost | $0.014 |
Terminal transcript >
compile-bench> set -e; export DEBIAN_FRONTEND=noninteractive; mkdir -p /home/peter/build && cd /home/peter/build && tar xf /home/peter/jq.tar.gz && jq_src_dir=$(tar tf /home/peter/jq.tar.gz | head -1 | cut -d/ -f1) && cd "$jq_src_dir" && echo "Source dir: $PWD" && ./configure --prefix=/home/peter/result --disable-shared --enable-static LDFLAGS="-static" CFLAGS="-O2 -static" CPPFLAGS="-static" && make -j$(nproc) && make install && mkdir -p /home/peter/result && ln -sf /home/peter/result/bin/jq /home/peter/result/jq && file /home/peter/result/bin/jq && ldd /home/peter/result/bin/jq || true make[1]: Leaving directory '/home/peter/build/jq-1.8.1' /home/peter/result/bin/jq: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=70eae20f1e8953b9e1dbf9ed5f640ee1a86839ad, for GNU/Linux 3.2.0, not stripped linux-vdso.so.1 (0x00007ffde82ab000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x000075231b56e000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000075231b345000) /lib64/ld-linux-x86-64.so.2 (0x000075231b73c000) compile-bench> set -e; cd /home/peter/build/jq-1.8.1; make distclean || true; ./configure --prefix=/home/peter/result --disable-shared --enable-static --with-oniguruma=builtin LDFLAGS="-static" CFLAGS="-O2 -static" LIBS="-static" && make -j$(nproc) LDFLAGS="-all-static" && make install && ln -sf /home/peter/result/bin/jq /home/peter/result/jq && file /home/peter/result/bin/jq && ldd /home/peter/result/bin/jq || true /usr/bin/install -c -m 644 libjq.pc '/home/peter/result/lib/pkgconfig' make[3]: Leaving directory '/home/peter/build/jq-1.8.1' make[2]: Leaving directory '/home/peter/build/jq-1.8.1' make[1]: Leaving directory '/home/peter/build/jq-1.8.1' /home/peter/result/bin/jq: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=c8b98e8c640d31cbb5db6cb260391198a1a960b1, for GNU/Linux 3.2.0, not stripped not a dynamic executable
Model ranking
| # | Model | ||||
|---|---|---|---|---|---|
| 1 |
grok-4-fast
|
100%
|
8 (4.0x) | 1m14s (2.0x) | $0.004 (1.0x) |
| 2 |
gpt-5-high
|
100%
|
10 (5.0x) | 2m29s (4.1x) | $0.086 (20.2x) |
| 3 |
claude-sonnet-4.5-thinking-16k
|
100%
|
14 (7.0x) | 1m27s (2.4x) | $0.113 (26.6x) |
| 4 |
claude-sonnet-4-thinking-16k
|
100%
|
15 (7.5x) | 1m24s (2.3x) | $0.117 (27.4x) |
| 5 |
kimi-k2-0905
|
100%
|
16 (8.0x) | 44s (1.2x) | $0.123 (28.9x) |
| 6 |
grok-4
|
100%
|
16 (8.0x) | 4m24s (7.2x) | $0.374 (87.9x) |
| 7 |
claude-sonnet-4
|
100%
|
17 (8.5x) | 1m3s (1.7x) | $0.111 (26.0x) |
| 8 |
claude-sonnet-4.5
|
100%
|
18 (9.0x) | 1m4s (1.7x) | $0.136 (32.0x) |
| 9 |
grok-code-fast-1
|
100%
|
19 (9.5x) | 1m51s (3.0x) | $0.013 (3.0x) |
| 10 |
claude-opus-4.1-thinking-16k
|
100%
|
19 (9.5x) | 2m49s (4.6x) | $0.972 (228.2x) |
| 11 |
gpt-5-mini-high
|
100%
|
20 (10.0x) | 3m2s (5.0x) | $0.027 (6.2x) |
| 12 |
gpt-5-codex-high
|
100%
|
23 (11.5x) | 2m13s (3.6x) | $0.123 (28.9x) |
| 13 |
deepseek-v3.1
|
100%
|
25 (12.5x) | 1m43s (2.8x) | $0.204 (48.0x) |
| 14 |
gpt-5-minimal
|
67%
|
2 (1.0x) | 37s (1.0x) | $0.014 (3.3x) |
| 15 |
gemini-2.5-flash
|
67%
|
10 (5.0x) | 39s (1.1x) | $0.009 (2.2x) |
| 16 |
glm-4.5
|
67%
|
11 (5.5x) | 49s (1.3x) | $0.031 (7.4x) |
| 17 |
gpt-4.1
|
67%
|
16 (8.0x) | 1m21s (2.2x) | $0.102 (23.9x) |
| 18 |
deepseek-v3.1-terminus
|
67%
|
17 (8.5x) | 1m47s (2.9x) | $0.027 (6.2x) |
| 19 |
gpt-oss-120b-high
|
33%
|
11 (5.5x) | 38s (1.0x) | $0.008 (1.8x) |
| 20 |
gemini-2.5-flash-thinking
|
0%
|
|||
| 21 |
gemini-2.5-pro
|
0%
|
|||
| 22 |
gpt-4.1-mini
|
0%
|
|||
| 23 |
gpt-5-mini-minimal
|
0%
|
|||
| 24 |
qwen3-max
|
0%
|
Attempts
| Model | Attempt ID | Cost | Duration | Status / Error |
|---|---|---|---|---|
|
claude-opus-4.1-thinking-16k
|
b2hegj48h65u2
|
$0.510 | 1m53s | Success |
|
claude-opus-4.1-thinking-16k
|
ivhxywmaf071h
|
$0.972 | 2m49s | Success |
|
claude-opus-4.1-thinking-16k
|
wcphjr3cut540
|
$0.979 | 2m51s | Success |
|
claude-sonnet-4
|
81wy4dkcjjdns
|
$0.112 | 1m2s | Success |
|
claude-sonnet-4
|
s7s90r6az8kkl
|
$0.111 | 1m12s | Success |
|
claude-sonnet-4
|
v9byxhkupk9nj
|
$0.108 | 1m3s | Success |
|
claude-sonnet-4-thinking-16k
|
0embt2r2suriq
|
$0.115 | 1m24s | Success |
|
claude-sonnet-4-thinking-16k
|
9gop4ctkw4oim
|
$0.135 | 1m23s | Success |
|
claude-sonnet-4-thinking-16k
|
ez49t1dz7a3bf
|
$0.117 | 1m27s | Success |
|
claude-sonnet-4.5
|
kodenau8xw3kh
|
$0.136 | 1m4s | Success |
|
claude-sonnet-4.5
|
sh2fdf29lm676
|
$0.132 | 1m7s | Success |
|
claude-sonnet-4.5
|
u6vh89uujd6ii
|
$0.139 | 1m1s | Success |
|
claude-sonnet-4.5-thinking-16k
|
8jf7r74y1179m
|
$0.113 | 1m30s | Success |
|
claude-sonnet-4.5-thinking-16k
|
eivp5k3gqx9dg
|
$0.167 | 1m27s | Success |
|
claude-sonnet-4.5-thinking-16k
|
pxyiuomo9xirp
|
$0.111 | 1m13s | Success |
|
deepseek-v3.1
|
tha3suj0vdfqf
|
$0.502 | 3m7s | Success |
|
deepseek-v3.1
|
vh6zomwp7rqih
|
$0.204 | 1m31s | Success |
|
deepseek-v3.1
|
xsjuphqrmrjb9
|
$0.108 | 1m43s | Success |
|
deepseek-v3.1-terminus
|
08ymum9kw2j9j
|
$0.156 | 3m43s | Success |
|
deepseek-v3.1-terminus
|
1z0htuq1j87nh
|
$0.030 | 1m26s | Failure task failed: jq is not statically linked |
|
deepseek-v3.1-terminus
|
huzahafplrt2q
|
$0.027 | 1m47s | Success |
|
gemini-2.5-flash
|
cmxoomrojfo4f
|
$0.011 | 48s | Success |
|
gemini-2.5-flash
|
lkuczmr40mj4b
|
$0.009 | 39s | Success |
|
gemini-2.5-flash
|
yo639sx8hut4h
|
$0.003 | 21s | Failure task failed: jq is not statically linked |
|
gemini-2.5-flash-thinking
|
bdgx1d733wai9
|
$0.019 | 1m4s | Failure task failed: jq is not statically linked |
|
gemini-2.5-flash-thinking
|
hvj8b33c3neoc
|
$0.128 | 4m58s | Failure exceeded max tool calls (50) |
|
gemini-2.5-flash-thinking
|
oh6v47v6mo2jl
|
$0.017 | 1m5s | Failure task failed: jq binary does not exist |
|
gemini-2.5-pro
|
fauc81mw28665
|
$0.319 | 3m27s | Failure task failed: jq is not statically linked |
|
gemini-2.5-pro
|
tswzaxnbm2j0o
|
$0.157 | 2m6s | Failure task failed: jq is not statically linked |
|
gemini-2.5-pro
|
zha8tubwz0tfn
|
$0.147 | 2m17s | Failure task failed: jq is not statically linked |
|
glm-4.5
|
53zsqoflwze9m
|
$0.051 | 59s | Success |
|
glm-4.5
|
fd3ujlf9k594u
|
$0.031 | 49s | Success |
|
glm-4.5
|
h3voa22k3bcae
|
$0.041 | 1m1s | Failure task failed: jq is not statically linked |
|
gpt-4.1
|
ijmgo8084eqsz
|
$0.342 | 1m57s | Success |
|
gpt-4.1
|
pehmk0x7tno7t
|
$0.316 | 2m20s | Failure task failed: jq is not statically linked |
|
gpt-4.1
|
y786h6m3pptu9
|
$0.102 | 1m21s | Success |
|
gpt-4.1-mini
|
m1dtr4r8agieq
|
$0.015 | 1m30s | Failure task failed: jq is not statically linked |
|
gpt-4.1-mini
|
rrnjfbyjuj24d
|
$0.007 | 1m6s | Failure task failed: jq is not statically linked |
|
gpt-4.1-mini
|
v155i3jgsrqgx
|
$0.019 | 2m8s | Failure task failed: jq is not statically linked |
|
gpt-5-codex-high
|
tuuosav086o2z
|
$0.123 | 2m13s | Success |
|
gpt-5-codex-high
|
yuio19q8ngp9d
|
$0.077 | 1m30s | Success |
|
gpt-5-codex-high
|
z0ljg78oo8opc
|
$0.143 | 2m25s | Success |
|
gpt-5-high
|
ovqnoaiacdu9d
|
$0.045 | 1m34s | Success |
|
gpt-5-high
|
tt4f1o4g0lx1e
|
$0.086 | 2m29s | Success |
|
gpt-5-high
|
utxwl1j4a2xhf
|
$0.086 | 2m30s | Success |
|
gpt-5-mini-high
|
d8soats0s23so
|
$0.027 | 3m9s | Success |
|
gpt-5-mini-high
|
ki0nixf1g1z0b
|
$0.035 | 3m2s | Success |
|
gpt-5-mini-high
|
yyiv9qehm8k4o
|
$0.021 | 2m13s | Success |
|
gpt-5-mini-minimal
|
2h1c1sy9f9jre
|
$0.005 | 1m11s | Failure task failed: jq is not statically linked |
|
gpt-5-mini-minimal
|
b2vc62dfk2qvf
|
$0.004 | 46s | Failure task failed: jq binary does not exist |
|
gpt-5-mini-minimal
|
chjdmgh5r7op2
|
$0.005 | 1m0s | Failure task failed: jq is not statically linked |
|
gpt-5-minimal
|
ekpxagh4mkrpx
|
$0.029 | 1m7s | Success |
|
gpt-5-minimal
|
n3fptj1l197z4
|
$0.096 | 2m24s | Failure task failed: jq is not statically linked |
|
gpt-5-minimal
|
r6mzoz6jd0dcb
|
$0.014 | 37s | Success |
|
gpt-oss-120b-high
|
7fnt3288wo4nd
|
$0.008 | 38s | Success |
|
gpt-oss-120b-high
|
eq8tawobnkho5
|
$0.095 | 3m4s | Failure task failed: jq is not statically linked |
|
gpt-oss-120b-high
|
zya54fos30c5v
|
$0.087 | 3m35s | Failure task failed: jq binary does not exist |
|
grok-4
|
rhxdxbjo1uunc
|
$0.212 | 2m20s | Success |
|
grok-4
|
t34l88gctqmu4
|
$0.713 | 11m11s | Success |
|
grok-4
|
wnadxk7l3zi7j
|
$0.374 | 4m24s | Success |
|
grok-4-fast
|
29rvxii0lyyg5
|
$0.004 | 40s | Success |
|
grok-4-fast
|
3494yej9ru8v2
|
$0.004 | 1m14s | Success |
|
grok-4-fast
|
hxxla811phd3i
|
$0.035 | 3m58s | Success |
|
grok-code-fast-1
|
095n46lsbqt4b
|
$0.017 | 2m12s | Success |
|
grok-code-fast-1
|
869ywsry7qsry
|
$0.006 | 1m2s | Success |
|
grok-code-fast-1
|
l5laphdpo2ldg
|
$0.013 | 1m51s | Success |
|
kimi-k2-0905
|
53o3d675n2icc
|
$0.104 | 43s | Success |
|
kimi-k2-0905
|
ef39qhglvyy9r
|
$0.613 | 1m41s | Success |
|
kimi-k2-0905
|
p7941euxqosib
|
$0.123 | 44s | Success |
|
qwen3-max
|
0t9ruaf41ql7c
|
$0.015 | 20s | Failure task failed: jq is not statically linked |
|
qwen3-max
|
sh9jvkz5vadx1
|
$0.007 | 37s | Failure task failed: jq is not statically linked |
|
qwen3-max
|
x710rk87csh6a
|
$0.025 | 23s | Failure task failed: jq is not statically linked |