Qwen 3.8 27B : l'environnement d'exécution surpasse le modèle dans les benchmarks de code

“Qwen 3.8 isn’t Opus level”: I re-ran the test. (Activity: 911): The image (link) shows the Deepseek/pi.dev-style coding harness being used with qwen3.8-27b in “Plan” mode for a C#/OpenGL ocean-rendering task, supporting the post’s claim that harness quality strongly affects observed model capability. In the author’s rerun, the same Qwen3.8 model and prompt failed under VS Code Copilot with a black screen, but succeeded under the alternate harness, reportedly using screenshot feedback and even generating a PNG decoder when vision was not enabled, producing waves, sky, sun, and underwater view in about 1 hour on an RTX 5090 running an ninfer-nvfp4 build with ~190k context at ~150–180 tok/s. Commenters largely agreed that the result demonstrates a large gap between “lazy” or sandboxed coding harnesses and agentic harnesses with execution/screenshot feedback. The original critic of Qwen3.8 conceded the prior conclusion was wrong and began retesting with pi.dev, noting fewer crashes and lower RAM use than VS Code/BYOM with llama.cpp.A key technical theme was that harness quality can dominate perceived model capability: commenters noted Qwen 3.8 apparently implemented an “on the fly PNG decoder” and still produced working ocean shaders despite an initially misconfigured or limited execution setup. The discussion framed this as evidence that sandboxed tools like Copilot-style environments may under-represent what coding agents can do when given a proper runtime/test loop.The original tester reported switching from VS Code + BYOM talking to llama.cpp to pi.dev after acknowledging the earlier harness was inadequate. They observed two concrete issues in the VS Code setup: driver errors after spawning the test executable, and random VS Code crashes while llama.cpp RocM 1200 build from Lemonade SDK continued running without output errors; by contrast, pi.dev had not crashed and used noticeably less RAM.Several commenters compared agent harnesses such as pi.dev/OhMyPi, opencode, and local llama.cpp setups, with interest in how much autonomy the harness provides beyond a standard Claude-like chat workflow. Hardware constraints also came up: users speculated that a RTX 5090 or similar high-end local GPU setup, potentially with tools like Ninfer, could make local agentic coding workflows more viable without cloud subscriptions.New qwen3.8:27b on a 39k line C to single-file HTML / three.js port (Activity: 655): A one-shot agent benchmark attempted to port a 2.1 MB / 39k-line / ~600k-token single-file C procedural shooter (skill-issue) into single-file HTML/Three.js, where the source was >2× the available 262,144 token context. On RTX 6000 Pro 96GB with vLLM, FP8 weights and FP8 KV cache, Claude Code + Opus 5 produced the only “okay” port in 21 min / 1759 LOC, while qwen3.8:27b via hermes took 4h18m / 949 LOC and via codehamr (repo) took 1h40m / 1056 LOC, both judged “bad.” Commenters suggested that direct “convert this code” prompts cause models to re-imagine behavior; a more reliable pipeline is to first generate a transpiler, get runnable target-language output, then iteratively rewrite function-by-function against high-level pixel comparisons or low-level register/state references. Technical debate centered on whether the poor local results were due more to prompt/harness design, missing decomposition/tests, or inference setup: multiple commenters warned that FP8 KV-cache quantization may significantly degrade long-context performance and suggested rerunning without it. Others argued the wall-clock gap is expected because Anthropic can parallelize across far more hardware, and recommended measuring vLLM tokens/sec, planning first, splitting the monolithic C file into modules, and adding behavioral tests before porting.Several commenters argued that direct “convert this codebase” prompting causes models to re-imagine the source rather than preserve behavior, even with frontier models. A suggested workflow is to first have the model help write a transpiler to the target language, then iteratively rewrite function-by-function while validating against high-level pixel comparisons or low-level register/value traces to reach pixel-perfect equivalence.Multiple comments questioned the inference setup, specifically FP8 KV-cache quantization, Q8, and not running the full bf16 Qwen 27B model on an RTX 6000-class GPU. The concern was that KV-cache compression/quantization could introduce severe quality issues for a long-context code-porting task, and that rerunning without FP8 KV-cache or with full bf16 would better isolate model capability from quantization artifacts.One technical explanation for the long runtimes was repeated KV-cache reprocessing in vLLM: if the engine releases the session cache, it may spend minutes recomputing prior context before generating any new tokens. A commenter suggested using LMCache to persist KV-cache in RAM, noting that cloud providers often avoid this latency by caching processed context across turns. Read more