Black October: your host will leak your AI key, so make the leak cheap
Wiz counts 14 dev platforms breached for stored env vars. Revoke at the issuer, rotate AI keys first, and scope every key so a stolen one is worth little.
Situation
01Since early October, developer platforms that hold customer environment variables have been breached one after another. Wiz has kept a running list since 30 October. By 7 November it had 14 names: Fireworks AI, Andromeda, Zeabur, Lightning AI, OpenCode, Railway, Warp, CommandCode and Northflank in October, then Brev (NVIDIA), Trigger.dev, Chainstack, kernel.sh and Blacksmith. RunPod has been named on social media but has published nothing, so it is not on our list.
The ways in differ. Wiz found no single vulnerability or dependency linking the incidents, and the root causes published so far range from stolen credentials to build infrastructure to a third-party vendor. What the public reports share is the target: stored secrets, and above all the ones that pay for AI model access. A stolen AI key turns into money faster than almost anything else in an env file. In the two cases Wiz traced, the first test call on a stolen key came within 30 minutes of the vendor breach.
This week the story moved past hosting. kernel.sh says an attacker downloaded data from its logging provider that held API keys and customer deployment secrets. Blacksmith, which runs GitHub Actions jobs, says someone got into the store that keeps CI job logs. So the lesson is not "pick a safer host". Anywhere your secrets sit in plaintext counts as a host, and that includes your logs. Assume one of them will leak, and make the leak cheap.
Facts
02- 01Wiz's tracker, first published 30 October and last updated 7 November, lists nine platforms that disclosed incidents in October (Fireworks AI, Andromeda, Zeabur, Lightning AI, OpenCode, Railway, Warp, CommandCode, Northflank), then Brev and Trigger.dev (2 Nov update), Chainstack (3 Nov), kernel.sh (6 Nov) and Blacksmith (7 Nov). It says all public reports highlight attackers "explicitly hunting stored environment variables, with a focus on those granting AI access", and that there is no single vulnerability or dependency linking them. Wiz notes RunPod only as social media reports.
- 02Wiz followed two enterprise AWS Bedrock keys whose theft it traces to one of the vendor breaches. Both got the same first call within 30 minutes of the reported credential access: a single InvokeModel from a residential IP, in a one-minute sweep that tested stolen keys across several organisations. For days after, each key saw only 14 to 120 calls a day from the same small Go client, in lockstep, which Wiz reads as one job checking the keys still worked.
- 03Case A: the key's policy allowed global.* cross-region inference profiles. Four days after validation the attacker probed the model catalog, mapped quotas, then made about 800,000 Bedrock calls in 30 hours before the key was remediated. About 90% succeeded and only about 6% were throttled, because global routing spread the load across regions. About three quarters of successful calls went to Opus models.
- 04Case B: the key allowed Bedrock only through us.* inference profiles. When it surfaced in an apparent reseller pool seven days after validation, thousands of calls failed over four hours because the pool asked for global.*. By the next morning a checker had found the working us.* model IDs, and one address then made about 1.7 million calls in 13 hours, over a million of them successful. Most of the rest were throttled by the account's regional quotas. Wiz puts the two cases together at likely over $100,000 at list price.
- 05Northflank says the first known decryption of secrets happened between 02:39 and 03:08 UTC on 30 October, and its first alert came at 11:43 UTC. It told customers to assume every environment variable was decrypted, to revoke credentials with the issuing provider rather than only replacing them on Northflank, and to rotate in this order: AI provider keys, cloud provider credentials, payment provider keys, private keys, then database and other service credentials. Its 6 November update adds IP restrictions and budget limits, and asks upstream providers to offer revocation APIs, IP restrictions and enforceable budget limits. It names a patched CVE-2026-98432 but has not described it.
- 06GitBook's 4 November postmortem describes a chain of forgotten credentials: an SSO login flaw let the attacker create staff sessions, which exposed the auth details of a misconfigured GitHub App, which opened internal repos and eventually a Cloudflare API key, which reached production secrets. GitBook says the attack targeted one customer, but it treats all customer data as exposed, including API tokens, integration tokens and secrets, and session signing secrets. It was offline for about 16 hours and told GitBook MCP users to re-authenticate.
- 07kernel.sh says a compromised Kernel API key was used to probe on 24 October and to reach a staff login at its logging provider on 27 October. The data downloaded from that provider "contained a small number of kernel API keys and app deployments' third-party secrets". On 3 November the same party used a hosting provider credential to reach four GPU servers and tried to start cryptominers. Kernel puts the impact at 0.28% of customers.
- 08Blacksmith says that on 5 November it detected and contained unauthorized access to the system that stores the console output of CI jobs run on its runners. It affected job-log data for about 2.6% of customer accounts, CI execution was not affected, and those customers were contacted directly. Blacksmith has not said whether secrets were in the exposed logs. A full report is pending.
- 09GitHub's own Actions guidance says automatic log redaction "is not guaranteed". Redaction mostly relies on an exact match, so structured secrets (JSON, YAML) and transformed values (Base64, URL-encoded, a JWT signed with a private key) can slip through unless registered as secrets, and error output can carry secrets into logs. If an unredacted secret reaches a run log, GitHub says to delete the log and rotate the secret.
- 10Trigger.dev (1 Nov) says an attacker accessed environment variables for a subset of organisations and that it has emailed affected owners. Chainstack (3 Nov) says access ran from 26 to 31 October through a third-party component, exposed account data and password hashes, and that it treats node credentials as exposed. Chainstack calls its incident "part of a broader AI-powered attack campaign".
- 11On attribution, the threads touch but do not join. kernel.sh believes its attacker used the AI-assisted pentesting platform ARTEX. Wiz found that an IP address Zeabur named in its own compromise also hosted an ARTEX page in October and later ran key checks on the Case A and Case B keys, but Wiz says it cannot definitively tie that operator to the theft or the floods. Wiz also found a second reseller checking the same keys, which points to a market in stolen keys, not one buyer.
Assessment
03We assess, with high confidence, that stored AI provider keys are the main prize in this wave, and with low confidence anything about who is behind it. Treat "one campaign" as unconfirmed. Fourteen platforms, at least five different published root causes, one vendor (Chainstack) calling it a broader campaign, one (kernel.sh) naming a tool, and Wiz finding at least two resellers testing the same keys. That reads like a market, with several crews selling into it, more than one actor. For builders the difference does not matter. A market means more people looking for your keys, not fewer.
The Wiz cases are the useful part, and they cut both ways. The scoped key in Case B did not save the account. It bought hours: thousands of failed calls before someone found the working model IDs, and regional quotas throttled much of the flood. It still ran over a million successful calls. So scoping is not a defence on its own. What it does is give you time to see the attack and kill the key before the flood. In both cases the attacker sent a test call within half an hour and then ran a steady check for days. Anyone watching per-key usage had days of warning.
That leads to the point most incident notices bury: rotating in the host dashboard does nothing. The key that leaked is still valid at OpenAI, Anthropic, AWS or Stripe until you revoke it there. Northflank says it plainly. Its customers are now asking AI providers for what cloud keys already have: revocation APIs, IP restrictions and hard budget limits.
The CI-logs angle is the new part this week, and we hold it carefully. Blacksmith has not said secrets were in the logs it lost. Kernel's logging data did hold secrets. Either way, a log store is a secret store if anything you run prints a token, and GitHub itself says redaction is best effort.
Implications
04- Your host is not the only copy. Every platform that injects your secrets into builds, jobs or functions holds them in a form it can decrypt. Count each one when you work out your exposure.
- Rotation means revoke at the issuer. A new value in the host's env settings while the old key still works at the provider closes nothing.
- Order matters because monetisation speed differs. AI provider keys get tested in minutes and resold in days. Cloud keys are next, then payment keys, private keys and database credentials.
- A scoped AI key is a delay, not a wall. Region or model limits, per-key quotas and budgets turn a stolen key into a slow, noisy one that you can catch.
- The validation trickle is your warning. A single odd call followed by a few dozen calls a day from one client is the pattern Wiz saw before both floods.
- CI logs, log drains and observability vendors are secret stores if anything prints a token. Structured and transformed secrets are the usual way past redaction.
- Integration and MCP tokens count. GitBook made MCP users re-authenticate, and a host's own API token can usually read every secret it holds.
Recommended action
05This week, list every place an AI provider key lives: hosts, CI, job runners, log drains, local env files, MCP configs. For anything on a platform in the Wiz list, revoke the key at the provider first, then issue a new one, then update the host. Go in Northflank's order: AI keys, cloud credentials, payment keys, private keys, database credentials. Then make the next leak cheap. Give each environment and each service its own AI key. Set a hard budget or quota per key or project wherever the provider allows it, plus a spend alert well below it. On Bedrock, limit each IAM policy to the regions and models you actually call, and avoid global.* profiles you do not need. Add IP restrictions where offered. Watch per-key usage for one stray call followed by a steady trickle. In CI, grep recent job logs for key prefixes, register any derived or encoded secret so it is masked, stop printing env, and move cloud deploys to OIDC so there is no long-lived key to log. Last, rotate your host's own API tokens and re-authenticate MCP and integration tokens.
What to watch
06- Blacksmith's full report, and whether the exposed job logs held secrets.
- Root-cause write-ups from Northflank and Railway, and any public detail on CVE-2026-98432.
- Whether RunPod confirms or denies an incident, and further additions to the Wiz tracker.
- Any vendor or researcher that ties the incidents to one actor with shared indicators, or rules it out.
- Whether OpenAI, Anthropic and AWS ship the revocation APIs, IP restrictions and hard per-key budgets that Northflank asked upstream providers for.
Sources
07PRIORITY / SECURITY / Wiz tracker (7 Nov): 14 AI developer platforms breached since early Oct, stored env vars hunted for AI keys. New: kernel.sh (secrets in a logging store), Blacksmith (CI job logs). Unscoped Bedrock key: ~800k calls in 30h. Revoke at the issuer, AI keys first. One actor unconfirmed. / TTB 1006