Stability AI's release of its latest major model, Stable Diffusion 3 (SD3), is facing significant backlash from the AI community due to its restrictive licensing terms, a growing controversy that prompted a major Stable Diffusion community hub to ban SD3-related work on Monday.
SD3 users are concerned about the potential implications of the SD3 license terms on future model development and improvement, which traditionally rely on open-source community contributions and involvement. Stability AI now separates out a consumer or a “Creator License” that is available only to developers with less than $1 million in annual revenue, less than $1 million in institutional funding, and less than 1 million monthly active users.
The license also limits the number of images generated to 6,000 per month. Anyone who needs to exceed those limits must pay for an Enterprise License.
Navigating the situation becomes even more convoluted as developers begin to adapt SD3 to specific use cases.
“The key piece here is that a derivative work is created anytime you train any model, even if it's not SD3 on outputs from SD3," explained Kent Keirsey, CEO of Invoke, in a YouTube video. “This means that any model trained on outputs from SD3 could be considered a derivative work, subject to the licensing terms with Stability AI having rights over it.”
Decrypt asked for some clarification. "Per the license, derivative works are subject to the licensing agreement, requiring ongoing fees, and Stability asserts in the definitions that derivative works include those trained on “Outputs” of the model," Keirsey explained, "effectively - They are stating that you own the images, but if you train a model on them you agree that this merits a license fee."
CivitAI bans SD3
The situation has escalated to the point where CivitAI—the world's largest repository for Stable Diffusion resources—recently announced a temporary ban on every single resource related to SD3 to avoid potential legal issues with the license.
"Unfortunately, due to a lack of clarity in the license associated with Stable Diffusion 3, we are temporarily banning: All SD3 based models, All models or LoRAs trained on content created with outputs from SD3 based models. This includes utilities such as controlnets," the statement reads. "The concern is that from our current understanding, this license grants Stability AI too much power over the use of not only any models fine-tuned on SD3, but on any other models that include SD3 images in their datasets."
CivitAI said that they are seeking legal counsel and reaching out to Stability AI for further clarification on the licensing terms.
Right now, the model and resources remain available on Stability’s own Huggingface repository. The SDXL+SD3 merges and additional resources that were hosted on Civitai’s servers, however, are not available anywhere else right now.
As a result, Civitai is encouraging users to either download SD3 from Huggingface or keep developing either new models and technologies based on SDXL or even see at other alternatives not developed by Stability AI. "All model creators should be aware of the terms they're agreeing to with SD3 prior to engaging with it," Civitai sated, "additionally there are other core models coming on to the scene that show great promise without such restrictions.
Some interesting generative AI models not developed by Stability AI include Pixart Sigma, Kandinsky, Lumina, Playground and Hunyuan. The community is already showing increasing interest in some of these alternatives with Playground and Pixart ranking higher than SD3 on Imgsys.
Kersey, however, is not really interested in these models. "I think the community at large is disappointed with SD3, but there’s not a motivating reason to move towards Pixart Sigma and Kandinsky. These models were already released, and if they were significantly better than SDXL, I believe we’d likely have seen that movement already," he told Decrypt, "instead, most of what we’re hearing is that many will continue to leverage the improving SDXL ecosystem."
Open source or “open source”?
Other major names in the Stable Diffusion community are raising red flags. The developer of Pony Diffusion, a popular non-photorealistic character-focused model, has also voiced concerns about the lack of clear communication from Stability AI regarding commercial licensing.
"Regrettably, the ambiguous rollout of SD3's commercial licensing [has] been quite disheartening,” the Pony Diffusion developer stated. “The lack of clear and proactive communication from Stability AI, especially concerning the new model's commercial use, has left me in the dark as only the non-commercial license of the model was mentioned in [the] initial release announcement."
The developer said he would “gladly acquire” the new Enterprise License mentioned by Stability AI to access higher limits, but after they “reached out to Stability AI the day the new commercial license was pre-announced... I have not received any acknowledgment or information."
Invoke has also been a victim of Stability's silence. "Despite attempts to engage, we have not been given access to an Enterprise license, so we can't comment on the specific terms," Keirsey told Decrypt, "it is likely that these licenses are negotiated individually, as is common with most Enterprise SaaS software licenses," he assured.
Although some Stability AI staffers have provided informal, non-binding replies assuring users that anyone can fine-tune the model, the overall phrasing of the licensing has developers worried.
For example, the license also operates on a recurring fee basis, meaning that it is active for one month and then ends if the fee is not paid. This could be a problem if other models trained with SD3 outputs are considered derivative works and are thus part of Stability AI’s license. If so, the fee could be perpetually assessed, or the developer's work would need to be destroyed.
"Upon termination or expiration, each party will return or destroy (or in the case of electronic information, render practically inaccessible) the Confidential Information of the other, including Stability’s Software Products and any Derivative Works,” the license reads.
This stipulation also raises concerns about intellectual property ownership and the ability to build upon existing models, a cornerstone of open-source development.
— Ostris (@ostrisai) May 14, 2024
Keirsey zeroed in on the potential conflict with open-source principles.
"It's a restrictive license, and it doesn't solve the underlying value proposition of what open source AI is intended to solve for professionals, which is you own your data you own your intellectual property—and the fine-tuned models that you might create from this—which are derivative works," he said in a YouTube video.
He revealed that a “large majority” of enterprises that have been in communication with Invoke are thinking of staying with SDXL, the predecessor to SD3. "Most of our clients plan to continue training and deploying SDXL models within their teams. It's evident that the community is not embracing SD3 either," he told Decrypt.
Decrypt contacted Stability AI, asking for clarification, but the company said it would be unable to respond in a timely manner. This article will be updated if they subsequently provide additional comment or information.
SD3’s uncertain future
As Keirsey noted, with the latest model from Stablility AI in legal limbo, some developers may opt to stick with SDXL and wait for whatever comes after SD3. Aggressive censorship in SD3 and its inability to generate certain human poses have also played against it.
"If the model remains closed and under restrictive licensing, it's unlikely that the community of model finetuners and commercial businesses that contributed to SDXL's success will transition to SD3," he told Decrypt, "It’s clear based on the API outputs that there is a larger and more robust model that has not yet been released, and that may prove to be a more interesting point of discussion."
With Stability AI reeling from the departure of its CEO and layoffs in recent months, its desire to increase revenue to continue operations is understandable. But the need for the company to explain its new SD3 license and how it impacts the work of its large community of developers is pressing.
The controversy has potential implications for the broader AI community and the development of open-source AI models. Nonetheless, SD3 has been downloaded over 2.7 million times from HugginFace —more popular than SDXL Turbo and Stable Cascade—and had over 34,000 hits on Civitai before the ban.
Editor's Note: This article has been updated to include comments from Kent Keirsey, CEO of InvokeAI.
Edited by Ryan Ozawa.