Add "Contains Generative AI Content" tag to the Content Gating system
~Nanochip
With the wave of generative AI content used throughout games/platforms that run on user-generated content such as VRChat, it should be required that creators tag their avatars & worlds appropriately with "Contains Generative AI Content" when the avatar/world contains ANY generative AI assets/content.
The goal is so that users who do not wish to see such content, can filter it out through VRChat's native Content Gating system.
Screenshot below as an example.
Photo Viewer
View photos in a modal
Log In
Heather May
What exactly is the threshold for "This (Avatar/World) Contains Generative AI Content"?
Does it apply when:
* A single AI-generated texture is used?
* AI-assisted Udon code?
* AI-assisted writing or translations?
* AI-generated meshes or textures throughout the world?
* Or only worlds that are primarily AI-generated?
Without a clearly defined threshold, this could apply to almost any modern world. Where is the line drawn? A common response is "if I can see AI, it's too much." But if the concern is generative AI itself, shouldn't that also include invisible uses like AI-assisted code, writing, or development tools?
Does first vs third party make a difference? Does the creator have to use AI themselves, or does it also count if they include a third-party prefab that contains AI-generated content?
There are popular, largely handcrafted worlds that have contained small amounts of AI-generated content for years without much controversy because it fits naturally within the world. That feels very different from "This world is almost entirely AI generated". Similarly, I was in a world from last year that also used AI content and while people commented on it, it was a small part of the world and felt appropriate in how used.
This isn't the same as age gating, though. This isn't "inappropriate to minors" - indeed, the complaint is not about minors seeing it, but about you (and others) that I assume are adults seeing it. A disclosure on the world page makes sense, but that's very different from treating it like an age-gated content warning. If you use that as a binary "do I go or not" you will be excluding some fairly well-made worlds that may contain a small amount of AI - including worlds you may not expect.
The Steam approach - a notice, but not a restriction, that visible AI is present - may be the approach closest to what you're asking, but it doesn't really answer the question of "was generative AI used". If your complaint is the rise of AI use, it's in far more than the visual arts.
AI generated thumbnails may be the best indication: If they used AI for their thumbnail, they probably used it liberally in the world as well.
* Disclaimer: AI was used in the writing of this post. I am known to ramble, and I used it to help me tighten this post up
Yewnyx
This raises a lot of fundamental issues, none of which the OP bothers to tackle:
- How is something determined to be AI-generated?
- Who decides if something qualifies to be tagged as AI-generated?
- What is the punishment for failing to tag something that is AI-generated?
- What is the appeals process when someone has been punished for AI generation when they have not actually done so?
- How is the appeals process resilient against abuse by people who did use AI generation?
- Does a voluntary no-punishment tagging system have any impact at all?
- Does implementing such a system create social fallout, i.e. creating content police that raise accusations of AI generation whether or not any has been done?
- Does implementing such a system create pressure to add expand filters according to personal preference?
- Could such an expansion of filters lead to discriminatory behaviors?
I personally don't AI-generated images to be particularly interesting, but I do find this proposal lazy and incendiary, because it doesn't suggest any possible way to enforce this that avoids a ton of bullying and witchhunts. This is an obviously terrible idea, not because the intention is wrong, but because there isn't any suggested solution here that doesn't have the potential for massive abuse. Without concrete measures to ensure the medicine isn't worse than the poison, I don't think this is something that could ever actually be put into practical effect. This would be an absolute nightmare for an organization to actually put into practice.
Agdistis
Yewnyx
I find it so... interesting that you come out of the gates swinging.
I agree there are issues not addressed, but to come out and say it's "lazy and incendiary" is disingenuous and hurtful, for the sole purpose of putting yourself on a moral pedestal above nanochip in an attempt to claim your opinions carry more weight than theirs.
Ideas are just that. Ideas. Simply because something is incomplete does not mean it is evil or objectively bad; it simply means it is incomplete. With the rise of AI, it was inevitable that such a discussion would eventually be had, and I see this as the start of that discussion. For reasons unbeknownst to me, you've chosen to squash it.
Lastly, you're operating under the assumption that such a system would be policed and that consequences would be delivered. This can be inferred from the original language, "it should be required that creators tag," in the original post. But, given the overall accusatory tone of your comment, I'm unable to extend any benefit of the doubt.
Even if this post dies, the topic will come up again. I hope next time you'll be more open, considerate, and helpful.
-CMC-
Yewnyx Look at how Steam did tags AI generated content and allows people to know what uses AI and filter it. There's the solution. It's up to the game developer to justify its use from there. There's tagging and filtering on sexual content there and the world didn't end, same with crypto. If someone is using AI in their work, whether for coding or content generation, they should be able to justify its use with dignity if they stand by it. There should be different granularity levels, such as AI code, AI code and Gen content, AI Gen content only, etc. And the community should be able to decided whether they want to view or partake in that content. They could be fine with AI code, but they don't want anything to do with AI art, and that's totally fair. If someone is worried about the implications of people not liking the use of AI, perhaps don't use it, if the consequence of people finding out are something that can't be handled. One could also keep their head down and turn off social media if they can't handle the scrutiny. They already chose to use AI and may not need to justify anything, just tag their work. Fielding the social implications is semi-optional, ignore or block people criticizing and keep on going, might as well.
Yewnyx
-CMC- The analogy doesn't really hold, because Steam games are something you enjoy personally, rather than something you necessarily display. If a part of your "Blah is playing Foo" status update was "* and this game has AI content", and if you game status was completely public to anyone on the internet, someone might be able to say "get a load of this guy, playing a game that used genAI", even if you weren't particularly aware of it.
In social situations, one doesn't
have
to actually use or want to see anything generated for someone with enough social clout, or at least loudness with persistence, to hurl accusations and abuse towards someone on account of it, just like someone who isn't a minor with a high pitched voice might get yelled at for being a minor, whether or not they've age verified, whether or not they approve of age verification or the provider for it.Given the potential for abuse the question has to be asked whether the juice is worth the squeeze. For age verification, the answer is clearly yes, because there is a bright-line test that is objectively truthful and gets surfaced in a way that is extremely useful in adversarial situations: "is this person an adult?". Having a definitive affirmative answer is useful.
For genAI, as much as we might benefit from such a clear test, none exists, and the false negative rate is likely to be sky-high, elevating the bullying risk in a very obvious way. The problem cannot be solved in isolation from the social impact, and all solutions to moderating genAI cannot avoid the social, cultural, and policy implications that follow.
Agdistis Nice bait. No, a feature request that asks for a signal that something is immoral that does not bother to propose how to mitigate people being immoral as a direct consequence of that signal is not just incomplete, it is a witchhunt hazard. I think that being able to minimize one's exposure to pixels generated by AI is actually worthwhile, but doing so carelessly in an environment in which people easily bully each other is an act of harm and I am offended when people do rude things and avoid taking responsibility be cause the form was apparently polite. This proposal is obviously compelling enough to snowball yet too ill-considered to have any satisfying response from anyone.
-CMC-
YewnyxThis is important, like 18+. VRChat is given life by creators who have strong feelings about this subject, they should be empowered rather than disenfranchised. You talk as if dealing with negative social implications of AI are ‘inevitable’ in the same way that using AI is inevitable. People in this community who've come together to create solutions and come to a consensus on various matters regarding the last 10+ years of this game are going to suddenly loose their senses and turn into an overwhelming mob? VRChat has fielded community adversity before, and its our job as the reason this game continues to exist at all to scrutinize instead of accepting everything we're handed if we want to stay here happily. Most people are Grey, not B & W caricatures. There are certain things I'm totally fine with using AI code, but I'd rather filter AI art. I want the choice and agency to make that decision within this platform as I see fit.
People like Epic's Time Sweeny are upset both that users are vocally against him on social media regarding AI AND that Steam has AI tags. With UE6, he's tripling down on selling a fiefdom at the engine level to make his will of AI 'inevitable'. On the store front, Valve thankfully ignores his pathetic mewling. Consumers are happy with the labels, the world keeps turning. Speaking of abuse, people rail on Steam's review system because people use it for memes, comedy, and to spew toxic right wing nazi garbage and brigade every time a Kirkening-esque curator turns up due to GamerGate wastes of skin like Grummz saying things about women and minorities on X. The VAST majority of reviews are extremely helpful! Brigaded reviews reported by the community have been eliminated by Valve.
AI proponents who don't have the ability to buy out a captured audience should fight for what they believe in with their comrades if they think AI is worth it, and keep their heads down in the ways that I listed if fielding the social consequences is too much for them. If they want to target a skeptical audience, and it turns out they don't like the ethics involved, maybe that should've been considered. Obfuscation and normalization of a certain technology that is known to be HIGHLY unethical for ripping artists, ripping code, and ripping copyrighted material for over the last decade to the tune of Quadrillions of bytes? Give us tools to choose.
~Nanochip
Yewnyx You fail to realize like any other content tag, this obviously isn't a black & white tag, it's entirely a gray area and should be treated as such on a case by case basis.
The content gating system exists and would be a decent starting point/foundation for self-reported AI generated content that a creator publishes onto the platform. This way, users can filter it out if they decide to.
That said, a lot of your questions assume that this is the end-all-be-all solution in an ever-changing creative debate, which I absolutely don't have the energy to get involved in.
Regardless of your view, there's no doubt it should be up to the user where they define the line on gen AI content, but at the bare minimum, this proposal is a starting-point/foundation for the self-reported gen ai content.
What happens after this? I don't know, but I'd definitely consider ideas/canny posts later down the line!
As for your jab on feeling this proposal is lazy and incendiary, I hope you have a great day and don't dwell on this too much. 😊
Agdistis
Yewnyx
"I am offended when people do rude things and avoid taking responsibility"
Not sure why you think I'm trying to bait you, or with what.
I'm really not sure how you've managed to, in the same breath, acknowledge its incompleteness and still insist that VRC would push an incomplete feature.
"Doing so carelessly" - then offer some ideas or solutions. You've done nothing but kick and point fingers and exhibit the same behaviors you yet complain of.
Not sure why you're so angry, or why you think your anger will produce anything of value, but I promise you there are better ways to spend your energy.
Photo Viewer
View photos in a modal
Raergur
please. Artists are the only thing that gives VRChat its life. They need to be prioritized or I guarantee this platform will go downhill.
Sylan Troh
Personally I feel this is a serviceable idea. Sure you can't make people tag their content accurately, but that's no reason not to do it because the same applies for all the other content filters, and those definitely help people regardless of the fact that some content goes unreported. And there are absolutely enough people who do not want to see any generated content whatsoever that a feature like this is warranted.
I do still think that there may need to be a broader rethinking of the content gating system to deal with people who refuse to engage with it. It definitely happens with all of the tags, but in my experience, almost all people who generate content with "AI" specifically try to be deceptive about it because they know it's unpopular. So I suspect this tag in particular will be even more underutilized than the others. Not sure what a different approach to content gating would look like though, so I think in the short term, this makes sense.
IrbisTheSnep
All filters are to protect minors and to filter traumatizing content. Ai slop filter would be a filter for preferences/taste. Adding it would be close to adding filters for furries, lgbt etc. I think we need a different approach to this problem than adding another filter to existing list of nsfw filters. Community tags, similar to steam game tags, or other world scoring system would better address the problem here.
~Nanochip
IrbisTheSnep Absolute insane take comparing a Gen AI filter to "furries, lgbt etc" 💀
IrbisTheSnep
~Nanochip My sanity is not a topic of this discussion. Both LGBT and AI are topics of individual moral compasses. I think hating LGBT is bad, but its my morality compas to think that and I don't think about forcing people to like it either. Some poeple hate LGBT as strongly as AI is hated. Both topics have also supporters. This os why I would see option to give everyone an option to work with tags and give an option to make individual blacklist of tags. This way not only AI would be handled, but also other questionable content with phobias or fans, for example diapers, spiders, holes...
FemPyro
IrbisTheSnep Bro I'm sorry I might be tripping, but you type like an AI
IrbisTheSnep
FemPyro Lol. Someone in my workplace also told me something similar. Dunno if it's my age or way I learned english. I have quite scientific mind and for long time I struggled to write/share my thoughts in an easy to understand way, so I learned to use similarities to explain different points of view. I rarely use AI and mostly as a tool for initial research, to later check sources it used myself. My writing style did not change in last 10+ years, so it might be the other way, AI was trained to write like me. XD
IrbisTheSnep
I assure you I'm a hum... I mean furry. XD
ふあゆき
Well, seeing comments like that around here is exactly why I can't agree with this proposal. Some people are so desperate to turn this into a moral crusade against AI that they completely ignore basic logic. You need to be smart enough to handle the scissors before you start running around with them.
Churii~
fully agree, this should be a filter option. i do not like seeing worlds that look like one thing in the thumbnail bc of ai then are totally different looking when i load in.
Patroll
AI generated content has been greatly impacting my experience as a world explorer. I don't browse new worlds in-game anymore and instead look for good ones on VRC photographers social media. AI content doesn't help anyone. All it creates is a miserable experience for users, low quality content hidden behind fake thumbnails, content "creators" using AI and producing slop that confuses its users instead of learning how to create and design worlds that people want. It's a miserable experience where people who don't know anything about design, lighting, modeling, texturing etc. can now mass produce garbage that's only impressive if you know absolutely nothing. If I wanted to see shit I'd generate it myself. I browse worlds to see things with at least a single thought put into them. I want to experience CREATIONS not HALLUCINATIONS.
SaoGalaxy
Patroll that sounds like a rule change on how thumbnails are made then ai overall. just auto generate thumbnails from the ingame camera .
Axinovium
Nope. This would be incredibly short-sighted and counterproductive for an ecosystem that relies on user-created content. AI-assisted workflows empower ordinary people without lots of free time and money to create on an equal footing. AI assisted workflows are very obviously the future of gamedev and content creation.
A bad world is bad whether it was made by AI or by hand, and a good world should not need a warning label just because a texture, script or a prop was AI-assisted.
The real thing people dislike is low-effort, lazy worlds with thumbnails which don't represent them accurately. This was always a problem, even before AI. But it's one that is largely mitigated by the community labs system.
Content gating should be based on what the user actually experiences, not what tools the creator used. VRChat's filter categories describe the content itself. “Made with AI” describes a tool used.
Not to mention, it's also completely impossible to even enforce. How do you prove if something was AI assisted or not? You'd be surprised at how many big world creators are quietly using AI for their world development behind the scenes for scripts, textures, upscaling, audio effects, etc.
The gatekeeping is getting exhausting. Let creatives build with the tools they want to use.
Gamekiller48
Axinovium For some reason the "a good world shouldn't need an AI-assisted warning label" and similar sentiments only ever comes from people who tell everyone that AI is inevitable and can make the end result so much better, while at the same time fighting tooth and nail against having to disclose that they're using it.
No one is owed visits to their world, least of all people who outsource the creation process to tools that only work off of stolen work.
Add the tag, and then have it stand on its own merits, rather than allowing deceptive creators to succeed off of stolen valor.
Axinovium
Gamekiller48 Allowing it to stand on its own merits would mean not adding tags though and people judging a world for what it is rather than pre-deciding it based on their personal ideological beliefs.
Gamekiller48
Axinovium That would only be true if AI generated content was only an aesthetical and quality question, which it isn't: it's also an ethical question.
In that way it should be treated like all other existing products with ethical issues, such as cans of tuna that have to add the disclaimer if the fish inside was caught by rod or by dragnet, or meat in the supermarket that has to declare in which conditions the animals have been kept, or labels on chocolate that allow the customer to see if their chocolate was produced on plantations where the workers received fair wages or not.
The details of production which allow the consumer to decide if the product was produced in circumstances that are reconcileable with their own ethics
is
part of the merit by which the product needs to be judged by.SaoGalaxy
Gamekiller48thats old news , ai your talking about is not the same as world building since a ai world is not a ai picture. nore are they build the same way.
Axinovium
Gamekiller48 AI is only an ethical issue for people who don't understand it and have a completely incorrect interpretation of things like intellectual property rights and environmental impacts.
It's all been debunked before; it's all just journalist slop and sensationalism designed to push an ideological agenda.
Just be honest: in practice, if this filtering was added, it would simply be used as a way for AI haters to witch-hunt creators who are building art they dislike and disagree with on purely subjective ideological grounds.
Gamekiller48
Axinovium Okay dude, if you want to mindlessly parrot the talking points of AI grifters then sure, all the ethical and intellectual property questions are all solved and settled.
If you live in the real world, you might realize that A) intellectual property rights differ from country to country, and that the approach these AI companies have used (fair use interpretation) does not have legal standing in large parts of the world, and is even on shaky grounds in the US itself because it arguably violates the fairness stipulation/non-compete provision of the laws it bases itself on.
B) that ethical issues are absolutely raised by people who understand it, you are of course free to strawman those arguments away by saying that only people who don't understand the technology hold them, but you'd be wrong. Case in point: arguably a good chunk of the LessWrong community.
C) That such filters would be used for witch hunts is one interpretation, that
could
be true, but we have no proof for that. It's also possible that it will reduce the amount of friction, as it will allow people who don't want to interact with AI generated content to simply ignore these worlds.D) Because I know you'll just mentally earmark me as someone who "doesn't understand AI" at this point: I've developed AI systems of my own, have a bachelor's in computer science, wrote my thesis on data annotation tools, and had the lectures on ethics and property rights.
Correctly marking the contents of
anything
for what it truly is is always
the correct pro-consumer choice.WubTheCaptain
Load More
→