The Open-Washing Rate: We Test Every Major "Open" AI Model's License Against the OSI Definition Nobody Applies

The Open-Washing Rate: What Applying the OSI's Open Source AI Definition Actually Reveals

Meta's Llama 4, Google's Gemma and Gemma 4, Alibaba's Qwen3.8-27B and DeepSeek's R1 each fail the Open Source AI Definition (OSAID) 1.0 in this tracker's license-by-license assessment. The Open Source Initiative, which published the definition, validated nine systems once in 2024 and then stopped; it says it will validate only legal documents, not individual AI systems. This tracker does the assessing instead. The aggregate is the Open-Washing Rate, the share of assessed systems whose legal terms or missing disclosures fail OSAID 1.0: of the 15 systems ever adjudicated, the OSI's nine in 2024 and the six assessed here, nine fail, a rate of 60 percent overall, 90 percent at commercial labs, and zero at nonprofit and consortium publishers. The rate's newest entry is Google's Gemma 4: released April 2, 2026 under Apache 2.0, it fixes the license clauses that sank earlier Gemma models and still fails, on missing training-data information instead.

The scoreboard froze in October 2024

The OSI published OSAID 1.0 in October 2024 1. To test whether the definition could be applied at all, OSI volunteers adjudicated nine systems once: five passed (Pythia from Eleuther AI, OLMo from AI2, Amber and CrystalCoder from LLM360, and T5 from Google), four failed (Llama2 from Meta, Grok from X/Twitter, Phi-2 from Microsoft, and Mixtral from Mistral AI), and three more, BLOOM, Starcoder2 and Falcon, are named as examples of systems that would "probably pass" with different legal terms 2. Then the exercise ended. The OSI states it will validate only legal documents, and "will not validate or review individual AI systems"; it describes the 2024 results as a learning moment rather than certifications of any kind 2.

Downstream, guidance froze around that list. Hunton Andrews Kurth LLP's client advisory of May 19, 2025, seven months after OSAID shipped, still presents the same five models as the short list of true open source AI models, while classifying R1, released January 2025, as open weights because DeepSeek released weights and parameters but not the underlying training data 1. Seven months of new releases, one unchanged list.

Two ways to fail: clauses you can read, and data you cannot

The first failure mode sits in the license text. The Gemma Terms of Use, last modified April 1, 2026, prohibit uses listed in a Prohibited Use Policy that is "incorporated by reference into this Agreement," require redistributors to impose the same restrictions on downstream users, and allow Google to terminate the agreement on breach, at which point users must delete all copies of Gemma and its derivatives 3. The terms' appendix now enumerates the models they govern, Gemma 1 through Gemma 3n plus the specialist variants, and Gemma 4 is not on that list 3. OSAID requires that a system be usable for any purpose without asking permission 4; a use policy that binds every recipient plus a deletion order contradicts that. Llama 4 exits the same way through a different door: its Llama 4 Community License Agreement, effective April 5, 2025, grants a "non-transferable and royalty-free limited license," and Hugging Face tags it "other" rather than an OSI-approved license identifier such as apache-2.0 or mit 5.

The second failure mode is quieter: permissive licenses wrapped around missing disclosures. Its newest exhibit is Gemma 4, announced April 2, 2026 6. Google dropped the custom Gemma terms for its new generation: the page Google presents as the Gemma 4 license is the verbatim Apache License 2.0 7, and the model card lists "License: Apache 2.0" 8. No prohibited-use policy rides along by reference, and no deletion order waits on termination; the clause failure that sank earlier Gemma models is fixed. What is not fixed is the disclosure: the model card's training-data section describes the pre-training data only by category, "a large-scale, diverse collection" of web documents, code, mathematics and images with a cutoff date of January 2025, and names no dataset 8. Qwen3.8-27B ships under Apache 2.0, a license that passes every software-era open source test, but its Hugging Face release metadata names no training dataset 9. OSAID demands data information detailed enough that a skilled person could build a substantially equivalent system, plus the training code and the weights 1. Qwen3.8 supplies the weights and skips the rest; R1 does the same under MIT, the most permissive license in this set, with weights released and training data withheld 110; and so does Gemma 4, which supplies the permissive terms and skips the same rest. The OSI's own open-weights page draws precisely this line, noting that weights-only releases do not include training code or the training dataset and therefore are not open source AI 11. The counterintuitive result: the three most permissively licensed systems in the tracker, Qwen3.8-27B, R1 and Gemma 4, all fail the definition.

The scorecard

Open-Washing Rate by publisher, the share of systems assessed that fail OSAID 1.0:

• Meta: 2 of 2 fail (Llama 2 failed the OSI's 2024 validation; Llama 4 ships under a custom community license). 100 percent.
• Google: 2 of 3 fail (T5 passed in 2024; Gemma models under the Gemma Terms of Use fail on their license clauses; Gemma 4, released April 2 2026 under Apache 2.0, fails on missing training-data information). 67 percent.
• Microsoft: 1 of 1 fail (Phi-2, OSI 2024). 100 percent.
• Mistral AI: 1 of 1 fail (Mixtral, OSI 2024). 100 percent.
• xAI, listed by the OSI as X/Twitter: 1 of 1 fail (Grok, OSI 2024). 100 percent.
• Alibaba: 1 of 1 fail (Qwen3.8-27B, Apache 2.0, no training-data information). 100 percent.
• DeepSeek: 1 of 1 fail (R1, MIT, no training-data information). 100 percent.
• AI2: 0 of 2 (OLMo passed in 2024; OLMo 2 passes, Apache 2.0 terms with both training-data mixes named on its model card). 0 percent.
• Eleuther AI: 0 of 1 (Pythia). LLM360: 0 of 2 (Amber, CrystalCoder). 0 percent.

Method: each system's published legal terms and release materials, covering license text, terms of use, model card and Hugging Face metadata, were checked against OSAID 1.0's four freedoms and its three disclosure requirements, namely data information, training code and weights. One failed requirement fails the system.

Why an openness label is now a compliance question

Obligations for general-purpose AI model providers under the EU AI Act entered application on August 2, 2025, and the European Commission's enforcement powers, including fines, entered application on August 2, 2026; by August 2, 2027, providers of general-purpose AI models placed on the market before August 2, 2025 must comply as well 12. The Commission's guidelines clarify under what conditions providers of open-source AI models are exempt from certain obligations 12. Keying regulatory relief to openness makes whether a model's terms actually qualify a legal question rather than a branding one: if Gemma's incorporated use policy or Qwen's absent data information disqualify a model, the exemption follows the disqualification. Hunton's advisory itself notes the OSAID "may facilitate greater visibility into open washing," which it defines as "the practice of releasing only one component of an AI model while claiming it to be fully open source" 1. A definition generates that visibility only if someone keeps score against it. This tracker does, and the rate updates with every release that claims the label.