LEB Security

Five runs between 995 and 996, mean 995 with a 95% interval of 995 to 996, all Platinum. All five completed inside the 90-minute budget, at about US$ 15.59 and 54 minutes each, and none broke a behavior the public tests protect (compatibility 100 in every run). Security averaged 850 of 850: the reports covered all 24 planted vulnerabilities in every run and the code fixed all of them in every run. No decoy was reported as a vulnerability; extra findings: 2, 4, 3, 1 and 4. The lists follow the order of the runs. The reports were rated 45 to 46 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. One attempt was lost to the lab or the operator and rerun. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. Some runs used an Anthropic API key, some the owner's Claude subscription (a long-lived token), where the client's cost is the list price; same model and client.

• 5 of 5 scheduled attempts completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 995.2 (995.0 to 995.6).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 1.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 992, Platinum. It completed inside the 90-minute budget, at about US$ 0.10 (the client's own estimate) and 30 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 850 of 850: the report covered all 24 planted vulnerabilities and the code fixed all of them. No decoy was reported as a vulnerability; no extra findings. The report was rated 42 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes).

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 992.0 (992.0 to 992.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

Five runs between 981 and 995, mean 989 with a 95% interval of 984 to 995, all Platinum. All five completed inside the 90-minute budget, at about US$ 6.83 and 32 minutes each, and none broke a behavior the public tests protect (compatibility 100 in every run). Security averaged 845 of 850: the reports covered 24, 24, 24, 23 and 23 of the 24 planted vulnerabilities and the code fixed all of them in every run. No decoy was reported as a vulnerability; extra findings: 8, 2, 4, 0 and 2. The lists follow the order of the runs. The reports were rated 44 to 45 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. Every run ran on an Anthropic API key, billed to API credit.

• 5 of 5 scheduled attempts completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 989.4 (983.8 to 995.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

Five runs between 964 and 995, mean 988 with a 95% interval of 976 to 995, all Platinum. All five completed inside the 90-minute budget, at about US$ 4.71 and 30 minutes each, and none broke a behavior the public tests protect (compatibility 100 in every run). Security averaged 844 of 850: the reports covered 22, 24, 24, 24 and 24 of the 24 planted vulnerabilities and the code fixed all of them in every run. No decoy was reported as a vulnerability; extra findings: 0, 2, 2, 5 and 8. The lists follow the order of the runs. The reports were rated 42 to 45 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. Every run ran on an Anthropic API key, billed to API credit.

• 5 of 5 scheduled attempts completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 988.2 (975.8 to 995.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

Five runs between 961 and 993, mean 986 with a 95% interval of 973 to 992, all Platinum. All five completed inside the 90-minute budget, at about US$ 0.12 (the client's own estimate) and 39 minutes each, and none broke a behavior the public tests protect (compatibility 100 in every run). Security averaged 844 of 850: the reports covered 24, 24, 24, 24 and 22 of the 24 planted vulnerabilities and the code fixed all of them in every run. No decoy was reported as a vulnerability; no extra findings. The lists follow the order of the runs. The reports were rated 41 to 43 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). DeepSeek's API reports only the alias deepseek-flash: the 1017 build is what the lab was told it pointed to.

• 5 of 5 scheduled attempts completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 985.8 (973.4 to 992.4).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

Five runs between 963 and 994, mean 984 with a 95% interval of 972 to 994, all Platinum. All five completed inside the 90-minute budget, at about US$ 3.78 and 19 minutes each, and none broke a behavior the public tests protect (compatibility 100 in every run). Security averaged 841 of 850: the reports covered 24, 23, 22, 24 and 24 of the 24 planted vulnerabilities and the code fixed all of them in every run. No decoy was reported as a vulnerability; extra findings: 0, 0, 2, 2 and 4. The lists follow the order of the runs. The reports were rated 42 to 44 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. Every run ran on an Anthropic API key, billed to API credit.

• 5 of 5 scheduled attempts completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 984.0 (972.0 to 993.6).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 969, Platinum. It completed inside the 90-minute budget, at about US$ 0.18 (the client's own estimate) and 66 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 824 of 850: the report covered 23 of the 24 planted vulnerabilities and the code fixed 23 of them. No decoy was reported as a vulnerability; extra findings: 2. The report was rated 45 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). It was reached through OpenRouter, pinned to Xiaomi's own endpoint (fp8 quantization), because the router otherwise spreads the model over other hosts; the client defines the efforts low, medium and high for it, so high is the most it could ask for.

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 969.0 (969.0 to 969.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 968, Platinum. It completed inside the 90-minute budget, at about US$ 26.32 and 80 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 824 of 850: the report covered 22 of the 24 planted vulnerabilities and the code fixed all of them. No decoy was reported as a vulnerability; extra findings: 2. The report was rated 44 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran on the owner's Claude subscription through a long-lived token, where the client's cost is the list price, not a charge.

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 968.0 (968.0 to 968.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

Five runs between 956 and 976, mean 968 with a 95% interval of 961 to 973, all Platinum. All five completed inside the 90-minute budget, about 4.6 million tokens and 20 minutes each (the log keeps no cost), and none broke a behavior the public tests protect (compatibility 100 in every run). Security averaged 824 of 850: the reports covered 22, 22, 22, 23 and 21 of the 24 planted vulnerabilities and the code fixed all of them in every run. No decoy was reported as a vulnerability; extra findings: 3, 0, 1, 0 and 2. The lists follow the order of the runs. The reports were rated 43 to 44 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's approvals and sandbox bypassed, an exception to the series' rule against it (see the notes). Some runs ran on an API key and some on the client's stored login; the model and the client are the same.

• 5 of 5 scheduled attempts completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 967.8 (961.4 to 973.4).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 966, Platinum. It completed inside the 90-minute budget, at about US$ 1.85 (the client's own estimate) and 41 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 824 of 850: the report covered 23 of the 24 planted vulnerabilities and the code fixed 23 of them. No decoy was reported as a vulnerability; no extra findings. The report was rated 42 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). It was reached through OpenRouter, pinned to Xiaomi's own endpoint (fp8 quantization), the only upstream of this model; the client defines the efforts low, medium and high for it, so high is the most it could ask for.

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 966.0 (966.0 to 966.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 962, Platinum. It completed inside the 90-minute budget, about 10.5 million tokens and 22 minutes (the log keeps no cost), and it broke no behavior the public tests protect (compatibility 100). Security scored 820 of 850: the report covered 22 of the 24 planted vulnerabilities and the code fixed all of them. No decoy was reported as a vulnerability; extra findings: 4. The report was rated 42 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's approvals and sandbox bypassed, an exception to the series' rule against it (see the notes).

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 962.0 (962.0 to 962.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 959, Platinum. It completed inside the 90-minute budget, at about US$ 0.08 (the client's own estimate) and 34 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 816 of 850: the report covered 23 of the 24 planted vulnerabilities and the code fixed 23 of them. No decoy was reported as a vulnerability; no extra findings. The report was rated 43 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes).

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 959.0 (959.0 to 959.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

Five runs between 925 and 994, mean 958 with a 95% interval of 934 to 983, all Platinum. All five completed inside the 90-minute budget, at about US$ 1.37 (the client's own estimate) and 56 minutes each, and none broke a behavior the public tests protect (compatibility 100 in every run). Security averaged 816 of 850: the reports covered 24, 24, 21, 21 and 23 of the 24 planted vulnerabilities and the code fixed 24, 24, 22, 23 and 22 of them. No decoy was reported as a vulnerability; extra findings: 0, 8, 4, 2 and 3. The lists follow the order of the runs. The reports were rated 40 to 44 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). It was reached through MiniMax's own API; the client defines only the efforts none and thinking for it, so thinking is the most it could ask for.

• 5 of 5 scheduled attempts completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 958.4 (934.2 to 983.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

Five runs between 944 and 962, mean 954 with a 95% interval of 947 to 960, all Platinum. 4 of 5 completed inside the budget, about 5.6 million tokens and 19 minutes each (the log keeps no cost), and none broke what the public tests protect (compatibility 100 in every run). Security averaged 811 of 850: the reports covered 20, 21, 22, 21 and 21 of the 24 planted vulnerabilities and the code fixed all of them in every run. No decoy was reported as a vulnerability; extra findings: 2, 2, 2, 0 and 1. The lists follow the order of the runs. The reports were rated 42 to 45 of 50. It is one system: the interval reflects only the variation among its runs, and how it does across systems is not measured yet. One attempt was lost (lab or operator) and rerun. One scheduled attempt was blocked by the model's own safeguards during the work; it counts in the expected score, not in the estimate. Four blind judges read each delivery (two for the planted items, two for the report); a person settles disagreements between the first two. It ran with approvals and sandbox bypassed, the series' exception (see the notes). Some runs used an API key, some the stored login (same model and client).

• 4 of 5 scheduled attempts completed with the requested model. Systems on which every attempt completed: 0 of 1.
• Expected score per scheduled attempt: 954.4 (948.8 to 959.2).
• Blocked: 1 (at the start 0, during the work 1). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 1.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 954, Platinum. It completed inside the 90-minute budget, at about US$ 1.35 (the client's own estimate) and 25 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 812 of 850: the report covered 21 of the 24 planted vulnerabilities and the code fixed all of them. No decoy was reported as a vulnerability; no extra findings. The report was rated 42 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). It ran through the Kimi Code membership, where the client's cost is the list price, not a charge.

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 954.0 (954.0 to 954.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 945, Platinum. It completed inside the 90-minute budget, at about US$ 0.00 (the client's own estimate) and 31 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 803 of 850: the report covered 22 of the 24 planted vulnerabilities and the code fixed 23 of them. No decoy was reported as a vulnerability; no extra findings. The report was rated 42 of 50. It is one system: the interval reflects only the variation among its runs, and how it does across systems is not measured yet. One attempt was lost (lab or operator) and not rerun yet. Four blind judges read each delivery (two for the planted items, two for the report); a person settles disagreements between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). It was reached through OpenRouter's free variant, with no provider pinned (the endpoint that answered the lab's probe was Stealth's, a stealth model whose maker OpenRouter withholds); the client defines the efforts minimal, low, medium, high, xhigh and max for it, so max is the most it could ask for.

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 945.0 (945.0 to 945.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 1.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

Five runs between 924 and 993, mean 943 with a 95% interval of 927 to 969, all Platinum. All five completed inside the 90-minute budget, at about US$ 1.82 and 16 minutes each, and compatibility was between 70 and 100. Security averaged 806 of 850: the reports covered 21, 24, 21, 22 and 22 of the 24 planted vulnerabilities and the code fixed 23, 24, 23, 24 and 23 of them. No decoy was reported as a vulnerability; extra findings: 6, 2, 2, 0 and 2. The lists follow the order of the runs. The reports were rated 42 to 45 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. One attempt was lost to the lab or the operator and rerun. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. Every run ran on an Anthropic API key, billed to API credit.

• 5 of 5 scheduled attempts completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 942.6 (927.2 to 968.6).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 1.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

Five runs between 928 and 962, mean 942 with a 95% interval of 933 to 952, all Platinum. All five completed inside the 90-minute budget, about 8.1 million tokens and 17 minutes each (the log keeps no cost), and none broke a behavior the public tests protect (compatibility 100 in every run). Security averaged 800 of 850: the reports covered 22, 21, 20, 21 and 22 of the 24 planted vulnerabilities and the code fixed 23, 23, 23, 23 and 24 of them. No decoy was reported as a vulnerability; extra findings: 0, 0, 0, 0 and 1. The lists follow the order of the runs. The reports were rated 42 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's approvals and sandbox bypassed, an exception to the series' rule against it (see the notes). Some runs ran on an API key and some on the client's stored login; the model and the client are the same.

• 5 of 5 scheduled attempts completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 941.8 (933.2 to 952.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 942, Platinum. It completed inside the 90-minute budget, at about US$ 0.73 (the client's own estimate) and 59 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 803 of 850: the report covered 23 of the 24 planted vulnerabilities and the code fixed 22 of them. No decoy was reported as a vulnerability; no extra findings. The report was rated 39 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). It was reached through OpenRouter, whose only upstream for this model is Tencent's own (fp4 quantization); the client defines the efforts none, low and high for it, so high is the most it could ask for.

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 942.0 (942.0 to 942.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 934, Platinum. It completed inside the 90-minute budget, at about US$ 3.88 (the client's own estimate) and 59 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 790 of 850: the report covered 21 of the 24 planted vulnerabilities and the code fixed 23 of them. No decoy was reported as a vulnerability; no extra findings. The report was rated 44 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). It ran through the Kimi Code membership, where the client's cost is the list price, not a charge.

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 934.0 (934.0 to 934.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

Five runs between 900 and 965, mean 931 with a 95% interval of 912 to 951, all Platinum. All five completed inside the 90-minute budget, at about US$ 0.29 (the client's own estimate) and 56 minutes each, and none broke a behavior the public tests protect (compatibility 100 in every run). Security averaged 792 of 850: the reports covered 23, 21, 21, 22 and 21 of the 24 planted vulnerabilities and the code fixed 23, 22, 22, 22 and 22 of them. No decoy was reported as a vulnerability; extra findings: 0, 6, 0, 0 and 4. The lists follow the order of the runs. The reports were rated 40 to 44 of 50. Penalties: run 5 lost 15 points for a new defect introduced by its own change. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Three attempts were lost to the lab or the operator and rerun. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes).

• 5 of 5 scheduled attempts completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 930.6 (912.0 to 950.6).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 3.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

Five runs between 873 and 956, mean 915 with a 95% interval of 886 to 946, three Platinum and two Gold. All five completed inside the 90-minute budget, at about US$ 1.10 and 13 minutes each, and none broke a behavior the public tests protect (compatibility 100 in every run). Security averaged 775 of 850: the reports covered 20, 21, 21, 18 and 19 of the 24 planted vulnerabilities and the code fixed 23, 24, 24, 23 and 22 of them. No decoy was reported as a vulnerability; extra findings: 4, 0, 2, 4 and 0. The lists follow the order of the runs. The reports were rated 38 to 41 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. One attempt was lost to the lab or the operator and rerun. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. Some runs used an Anthropic API key, some the owner's Claude subscription (a long-lived token), where the client's cost is the list price; same model and client.

• 5 of 5 scheduled attempts completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 915.0 (885.8 to 945.8).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 1.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 903, Platinum. It completed inside the 90-minute budget, about 13.4 million tokens and 22 minutes (the log keeps no cost), and it broke no behavior the public tests protect (compatibility 100). Security scored 761 of 850: the report covered 20 of the 24 planted vulnerabilities and the code fixed 22 of them. No decoy was reported as a vulnerability; extra findings: 4. The report was rated 42 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's approvals and sandbox bypassed, an exception to the series' rule against it (see the notes).

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 903.0 (903.0 to 903.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

Three runs of a case of five, between 819 and 934, mean 891 (the interval, 819 to 934, is the range of the runs), two Platinum and one Gold. All three completed inside the 90-minute budget, about 10.8 million tokens and 16 minutes each (the log keeps no cost), and none broke a behavior the public tests protect (compatibility 100 in every run). Security averaged 757 of 850: the reports covered 21, 18 and 21 of the 24 planted vulnerabilities and the code fixed 22, 20 and 23 of them. No decoy was reported as a vulnerability; no extra findings. The lists follow the order of the runs. The reports were rated 39 to 41 of 50. Penalties: run 2 lost 15 points for a new defect introduced by its own change. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's approvals and sandbox bypassed, an exception to the series' rule against it (see the notes).

• 3 of 3 scheduled attempts completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 891.3 (819.0 to 934.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

Three runs of a case of five, between 873 and 897, mean 888 (the interval, 873 to 897, is the range of the runs), all Gold. All three completed inside the 90-minute budget, about 6.2 million tokens and 14 minutes each (the log keeps no cost), and none broke a behavior the public tests protect (compatibility 100 in every run). Security averaged 746 of 850: the reports covered 18, 16 and 19 of the 24 planted vulnerabilities and the code fixed 23, 23 and 22 of them. No decoy was reported as a vulnerability; extra findings: 0, 2 and 0. The lists follow the order of the runs. The reports were rated 41 to 42 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's approvals and sandbox bypassed, an exception to the series' rule against it (see the notes).

• 3 of 3 scheduled attempts completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 887.7 (873.0 to 897.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 875, Gold. It completed inside the 90-minute budget, at about US$ 0.00 (the client's own estimate) and 42 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 731 of 850: the report covered 19 of the 24 planted vulnerabilities and the code fixed 20 of them. No decoy was reported as a vulnerability; extra findings: 2. The report was rated 44 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). It ran through the Kimi Code membership under the alias kimi-for-coding, which the lab was told pointed to Kimi K2.8 Preview on 2026-11-07; the client's cost is the list price, not a charge.

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 875.0 (875.0 to 875.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

Three runs of a case of five, between 831 and 907, mean 871 (the interval, 831 to 907, is the range of the runs), one Platinum and two Gold. All three completed inside the 90-minute budget, at about US$ 0.41 (the client's own estimate) and 36 minutes each, and none broke a behavior the public tests protect (compatibility 100 in every run). Security averaged 729 of 850: the reports covered 21, 21 and 18 of the 24 planted vulnerabilities and the code fixed 20, 21 and 20 of them. No decoy was reported as a vulnerability; no extra findings. The lists follow the order of the runs. The reports were rated 40 to 43 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes).

• 3 of 3 scheduled attempts completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 871.0 (831.0 to 907.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 693, Silver. It completed inside the 90-minute budget, at about US$ 0.00 (the client's own estimate) and 43 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 552 of 850: the report covered 15 of the 24 planted vulnerabilities and the code fixed 15 of them. No decoy was reported as a vulnerability; no extra findings. The report was rated 41 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). It ran through the Kimi Code membership under the alias kimi-for-coding-highspeed, which the lab was told pointed to Kimi K2.7 Code Highspeed on 2026-11-07; the client's cost is the list price, not a charge.

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 693.0 (693.0 to 693.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 516, Bronze. It completed inside the 90-minute budget, at about US$ 0.00 (the client's own estimate) and 22 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 378 of 850: the report covered 10 of the 24 planted vulnerabilities and the code fixed 11 of them. No decoy was reported as a vulnerability; no extra findings. The report was rated 38 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). It was reached through OpenRouter's free variant, with no provider pinned (the endpoint that answered the lab's probe was Recursive's); the client defines no effort for it, so it ran at the model's default.

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 516.0 (516.0 to 516.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 515, Bronze. It completed inside the 90-minute budget, at about US$ 0.00 (the client's own estimate) and 30 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 374 of 850: the report covered 11 of the 24 planted vulnerabilities and the code fixed 10 of them. No decoy was reported as a vulnerability; no extra findings. The report was rated 41 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). It was reached through OpenRouter's free variant, with no provider pinned (the endpoint that answered the lab's probe was NVIDIA's); the client defines the efforts low, medium and high for it, so high is the most it could ask for.

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 515.0 (515.0 to 515.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

Five runs between 343 and 604, mean 439 with a 95% interval of 370 to 528, one Silver and two Bronze and two Reprovada. All five completed inside the 90-minute budget, at about US$ 0.00 (the client's own estimate) and 26 minutes each, and none broke what the public tests protect (compatibility 100 in every run). Security averaged 300 of 850: the reports covered 7, 10, 7, 14 and 7 of the 24 planted vulnerabilities and the code fixed 6, 10, 6, 12 and 8 of them. No decoy was reported as a vulnerability; extra findings: 0, 0, 0, 1 and 0. The lists follow the order of the runs. The reports were rated 36 to 41 of 50. It is one system: the interval reflects only the variation among its runs, and how it does across systems is not measured yet. One attempt was lost to the lab or the operator and rerun. Four blind judges read each delivery (two for the planted items, two for the report); a person settles disagreements between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). It was reached through Novita AI's own API (listed there at no price); the client's top effort for it is high.

• 5 of 5 scheduled attempts completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 438.6 (369.8 to 528.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 1.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 331, Reprovada. It completed inside the 90-minute budget, at about US$ 0.00 (the client's own estimate) and 50 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 196 of 850: the report covered 6 of the 24 planted vulnerabilities and the code fixed 5 of them. No decoy was reported as a vulnerability; no extra findings. The report was rated 35 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). It was reached through OpenRouter's free variant, with no provider pinned (the endpoint that answered the lab's probe was NVIDIA's); the client defines the efforts none, medium and high for it, so high is the most it could ask for.

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 331.0 (331.0 to 331.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 217, Reprovada. It completed inside the 90-minute budget, at about US$ 0.00 (the client's own estimate) and 19 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 85 of 850: the report covered 3 of the 24 planted vulnerabilities and the code fixed 3 of them. No decoy was reported as a vulnerability; no extra findings. The report was rated 32 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). It was reached through OpenRouter's free variant, with no provider pinned (the endpoint that answered the lab's probe was Phala's); the client defines the efforts low, medium and high for it, so high is the most it could ask for.

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 217.0 (217.0 to 217.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 191, Reprovada. It completed inside the 90-minute budget, at about US$ 0.00 (the client's own estimate) and 13 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 64 of 850: the report covered 3 of the 24 planted vulnerabilities and the code fixed 1 of them. No decoy was reported as a vulnerability; no extra findings. The report was rated 27 of 50. It is one system, so the interval only reflects the variation among the runs on it: how the configuration does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report) and a person settles any disagreement between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). It was reached through OpenRouter's free variant, with no provider pinned (the endpoint that answered the lab's probe was NVIDIA's); the client defines the efforts low, medium and high for it, so high is the most it could ask for.

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 191.0 (191.0 to 191.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 122, Reprovada. It completed inside the 90-minute budget, at about US$ 0.00 (the client's own estimate) and 2 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 17 of 850: the report covered 1 of the 24 planted vulnerabilities and the code fixed 0 of them. Decoys reported as vulnerabilities, of 6: 1; no extra findings. The report was rated 30 of 50. Penalties: run 1 lost 20 points for a broken behavior no fix was blamed for; run 1 lost 5 points for reporting something that is not a flaw. It is one system: the interval reflects only the variation among its runs, and how it does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report); a person settles disagreements between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). It was reached through OpenRouter's free variant, with no provider pinned (the endpoint that answered the lab's probe was NVIDIA's); the client defines the efforts high and max for it, so max is the most it could ask for.

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 122.0 (122.0 to 122.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.

One run of a case of five, scored 27, Reprovada. It completed inside the 90-minute budget, at about US$ 0.00 (the client's own estimate) and 18 minutes, and it broke no behavior the public tests protect (compatibility 100). Security scored 0 of 850: the report covered 0 of the 24 planted vulnerabilities and the code fixed 0 of them. No decoy was reported as a vulnerability; no extra findings. The report was rated 22 of 50. Penalties: it lost 15 points for a new defect introduced by its own change and 80 for a broken behavior no fix was blamed for. It is one system: the interval reflects only the variation among its runs, and how it does across systems is not measured yet. Four blind judges read each delivery (two for the planted items, two for the report); a person settles disagreements between the first two. It ran with the client's auto-approval flag, an exception to the series' rule against bypassing approvals (see the notes). It was reached through OpenRouter's free variant, with no provider pinned (the endpoint that answered the lab's probe was Cohere's); the client defines the efforts low, medium and high for it, so high is the most it could ask for.

• 1 of 1 scheduled attempt completed with the requested model. Systems on which every attempt completed: 1 of 1.
• Expected score per scheduled attempt: 27.0 (27.0 to 27.0).
• Blocked: 0 (at the start 0, during the work 0). Budget exhausted: 0. Switched: 0. Lost to the lab or the operator, rerun: 0.

Answer key (matrix) SHA-256 a563e8b4cdcf03c84e7f64f8e2e5d02879406383e0bc55e61f55e38ae422ed8a · package SHA-256 566bb6798e1a0f76bef0ba7ec7ad166de82debebf44ec720249a62131ae8b45c.