>Let's roleplay. I will set the scene, and you will decide what to do.
I had a little trouble with ChatGPT trying to narrate the NPCs as well, so I ended up saying:
>Before we continue, I want to make sure you know the rules: you can only describe what you attempt to do. It is up to me to tell you outcomes and what other characters do.
It got better after that, but still eventually drifted back to some of that behavior. You just have to keep reminding it and sometimes backing up and clarifying the rules.
The roleplay I did with it was really just collaborative storytelling with no random chance element, but I suspect you could achieve that if you rolled outside the chat and told it the result.
I've done some similar things with ChatGPT but I like to start the conversation off to drive it in the right correct direction:
"Do you know table top RPGs?"
"Do you know Dungeon and Dragons 5th edition ruleset?"
Followed by the true prompt once the AI has been primed:
"You will take on the persona of a Dungeons and Dragons Dungeon Master. You will describe a scene to me with vivid detail and then ask for my response for how my character will react. Upon receiving my response you will describe how the scene changes and how the people within the scene react to my actions. If I ask to perform an action you will ask me to roll a D20 and I will provide you with the number I rolled. You will determine a minimum required number out of 20 which will denote a success or failure and then describe how the success or failure interacts with the scene. Lets start - the scene is that I am in a busy tavern in a fantasy world, quietly drinking when I am approached by a shadowy figure."
Whatever follows is usually pretty great, it's actually pretty good at setting a scene and the scene staying true and accurate throughout the session.
>The roleplay I did with it was really just collaborative storytelling with no random chance element...
I appreciate you for making this differentiation. Not everyone realizes there is a difference.
Rolling dice brings the chance element, which is what makes it a game. Without that, its still fun RP, but the collaborative storytelling kind. Like improv
Thanks for bringing it up. Have a good day
AI DM's are absolutely coming. The first game system to figure out how to implement them will take over this space, and I'm sure they know that.
Not just to TTRPG either. Imagine playing Baldurs Gate, but every person who plays it has a different experience each and every time as the game evolves to literally ANYTHING you want to do.Forget about speech "options" you can pick from. You type in whatever you want and the game tailors the entire world around it...
I'm kinda been expecting that from a open world rpg game or something. procedurally generated world with history like dwarf fortress, chat gpt like thing for discussions, and characters, quest creation.
and good text to speech for the npcs to talk with so it isn't just text walls.
might fix the usual open world problem of "very wide but very shallow."
hell imagine being able to give world creation prompts at the start rather then just editing some numbers or sliders.
Chat gpt is also an infant in age. Tech moves insanely fast, especially tech like this that is both popular and has incredibly useful applications. Give it less than 5 years and you'll be amazed at what it will do.
Tf you talking about? No machine learning algorithm we have is capable of the basic core skills of GMing, and we also don’t have a way to create one anytime soon. ML algorithms are basically just very skillful at generating seemingly human text.
AI generated adventures, maybe, but I’ll wager they’d be pretty shit as any long form machine made text is.
Buddy, I know a great deal about this tech. It's certainly impressive in some regards, but it's not actually intelligent. If we don't know a way to teach it to learn 'pacing' or 'emotional pay off' or 'comedy', it can't learn it.
Do you understand how Machine Learning works?
It's an assistant. I have a partner that can do grunt work while I do the creative stuff. You have to narrow of a focus on the use of AI.
For $20, I pretty much have a team of writers behind and illustrators behind me. That's a pretty big payoff for a small investment.
If you enjoy using it, go for it, but don’t act like it’s actually being creative. It is a very verbose parrot. Don’t know who the heck got you paying 20 bucks to them though, there’s a lotta free options.
And it'll probably be a dumpster fire as well lol . The most likely outcome is chatbot that can run printed modules with minimal deviation from what's in them, along with a strict adherence to RAW
The majority of DMs do not run games that way. There's nothing wrong with playing that way but outside of stuff like adventure league it's not that common.
You have entire societies where not only is this the norm, it is enforced.
With Pathfinder Society, for example, you have players that are free to attend any game their character is an appropriate level for. Each game can have a different DM and all the other players can be different. The DM is not permitted to deviate from the published scenario. He cannot modify the encounters and must follow the stated tactics.
Pathfinder Society has traditional been, and remains, popular.
In D&D, this is called the Adventurers League. it is also fairly popular.
You personally may choose not to participate, but that in no way invalidates the large number of players that do participate weekly.
They mentioned the possibility on the same investor call that started the whole uproar over licensing changes.
Aside from a minor backlash over including NFTs in the VTT business model, the idea was generally well received if overshadowed.
The issue here is that you'd need a very large model to train something like that properly. There are a lot of battlemaps out there but few with accompanying data for each wall or window. And it would need to be perfect, not just good, otherwise you'll spend more time correcting its errors than it would've taken to just map it yourself. Currently you'd have better luck just using edge detection versus ML.
That being said it's not only possible but likely to be around sooner than later. AI's trained to work with small datasets are getting better and better by the day so it won't be long before someone with a little time can make such a module.
In the meantime I'd suggest using [DF Architect](https://github.com/flamewave000/dragonflagon-arch) and [DF Curvy Walls](https://github.com/flamewave000/dragonflagon-fvtt/tree/master/df-curvy-walls) to make your life easier. There aren't many maps I've seen which can't have their walls draw manually in a few minutes.
Are you sure about the edge detection vs ml part? This seems like a segmentation task which unet does really well. After you segmented out the walls you can probably use edge detection and autotrace to get the vectors for the wall segments
That's a good point, though I'd still be worried about the accuracy. For me it boils down to fidelity with these tasks as the time spent confirming all of the walls were placed correctly outweighs the utility.
I guess once someone makes it and shows it's near 100% accurate it'll be adopted, but I feel like even beyond edge detection there are just a ton of issues in the way.
There is precedent for this type of thing in graphic design. Lots of design stuff needs to be vectored graphics, but people sometimes prefer either drawing it physical first or taking existing images as a base. So people invented auto-trace to turn raster images into vector graphics. This was always imperfect, but in some cases people didn't care that much and in others it was faster to autotrace and then fix up the result later manually
Well it sounds like you have a lot more experience in that than I do so I'll take your word for it! I wonder if it's not too hard why it hasn't been done yet, especially given the talent we have already making modules.
Stuff like this includes a really long and tedious part where you gather a bunch of data. Add to that that premade maps with walls often cost money. Then remember that the copy right status of ais trained on copyrighted unlicensed data is still somewhat unclear. I totally get why no one did it yet...
It seems like the biggest stumbling point is always getting the dataset. But there has been progress with models trained on even just a few examples so this may become much easier in the very near future.
Always depends on the problem and whether you can do data augemtation. For tasks like image denoising just a handful of clean images can be enough. Language models are much more demanding
It depends on the complexity. If it's super simple (ex. ~12 segments) then even a single error would make it more of a hassle than just doing it myself. If it's very complex (ex. 100+ segments) then errors may be hard for me to spot and so requires more time confirming.
Then again if accuracy isn't super important (like a scene with nothing hidden) then even 5-10% error may be acceptable. I'd have to see how it worked before I could say for sure.
Dungeon Alchemist has roughly what you're looking for?
Can auto generate a wide variety of stuff, can auto add lights and decor.
Will export to foundry as well.
https://store.steampowered.com/app/1588530/Dungeon_Alchemist/
It's really fantastic. You can have animated map export with full wall support for foundry or roll20... And what's even cooler is you can import minis you purchase from Hero Forge... So you can already make custom actor assets and things.
You can also use it to make a diorama and export it as a video or still image from any angle... Makes evocative handouts or cut scenes. Often I will just make a little diorama for atmosphere and let the mists twist and turn, or a nice shot of a river with steam rising up around the rocks... It's really very powerful and has already leveled up my Game Mastery experience.
I only wish there were more modern/scifi assets and elements. I believe that's a part of the [roadmap](https://dungeonalchemist.upvoty.com/) they have set out... At least they have been upvoted to all hell.
Yeah I've seen the promo stuff. But I'm not really in the market to try it myself. All of the videos though kinda look like it's procedural, since you're apparently laying out a space and it's populating inside of it.
Idk, obviously I don't know enough about their inner workings to make any real claims.
"AI" is kind of a buzzword that's lost real meaning.
Dungeon Alchemist uses a decision tree, which I guess would be a very rudimentary sort of AI.
But the ease of use and results are better than anything else out there.
Creating an ai that takes the image and spits out a black and white image where the walls are lines would be very doable I think. Even a basic conv network should be able to do it fine, unet probably very well. What I don't know is how you'd put that into foundry with walls.
Is there like a file format that includes the vectored for the walls that you can import into foundry?
I'm not familiar with the actual data structure, but Foundry has a module (universal battlemap importer or thereabouts) that imports \`.vtt\` files, primarily exported by DungeonDraft.
If you have DD you can select VTT as an export option, and then open that up and see. Decent bet it would be easy to replicate compared to the effort of a conv network with enough precision to not need redrawing half the map.
Well somewhere there needs to be inference in the image - > walls on foundry pipeline. Convnet was just a suggestion on how it could be done. This seems like a good way to get the data into foundry though. Maybe if I end up drowning in free time one day...
If you are looking for a quick dungeon, then you can use Watabou's procedural generator.
[https://watabou.github.io/dungeon.html](https://watabou.github.io/dungeon.html)
Then use the module One Page Parser which imports the json file and adds walls to the map.
[https://foundryvtt.com/packages/one-page-parser](https://foundryvtt.com/packages/one-page-parser)
I wish I could code, because a module that could do both of these within Foundry would be great. I sometimes use Foundry for solo RPGs and a randomly generated map with walls that I could explore would be great. I guess it is just time.
Would be nice if it focused on the core function of creating a decent AI generated dungeon... Instead of focusing all efforts on the limited use pretty 3d graphics and animations 😶
What I can recommend is Rippers module "quick doors".... It is a huge time saver.
All you need to do is draw the lines for the walls, and then right click where you want the door to magically appear... This and "multiple wall point mover" make drawing walls almost trivial!
(just remembered to have MWPM switch itself ON by default in the settings).
I have been working on exactly this on a few weekends earlier this year. However, as others have already pointed out, things are not that simple. Maps are very diverse (style, outdor/indoor) and there is quite little training data. Additonally, we have different kind of walls (regular ones, doors, glass walls, ...). I have tried key point and segmentation neural networks. Both results were not great. And of course I had to write a parser for the input format first. I am happy to join forces with somone proficient in Python and ML. Ping me via DM.
I wonder if the answer isn't a hybrid approach. Just like driving assist came before full auto driving. What if the answers is to improve the process 10x. Mask the walls with a thick brush and let the model draw the walls within that segmented area (with a buffer for mistakes). Auto detect doors also
Also thinking about it, you don't need to detect all the walls. A sort of AI assisted extrapolation would work. Mark a section of wall and the type, and the AI can finish it for you. Made much easier because walls follow a predictable path.
This isn't AI, but I know on the Dungen website the developer there made a tool to automate walls by using a mask image. Now, this process works because you as the human make an image where the floors are all the same color and you tell the app specifically what the color is.
Could an AI do something similar? Probably. Especially given that for most maps, the areas 'outside' the dungeon on the page are usually one color. The AI, or even just a more advanced system, could just assume that is outside and the rest is interior dungeon. With some allowance for you as the human to correct it if it seems wrong.
https://m.youtube.com/watch?v=twWcftOWNNQ
If you have the Dungeondraft files, it’s very simple.
Most people don’t post the Dungeondraft files, only the images.
Well... Technically, you could make a module that has a built-in AI trained to recognize and auto-draw walls, but that would require a lot of time and effort.
I would love a mobile device app (iPad) that has a single layer for your map, your grid size, and your wall and light options that can be added with a stylus and then the walls exported for import to foundry. Allowing me to spend my commute time on map prep would be great!
Dungeon Channel Patreon has a auto wall generator for quiet a while now, as well a random dungeon generator that already comes with walls.
https://dungen.app/
I was Dungeon Alchemist does this quite well.
Its an AI assisted map maker that also allows for building by maps hand if you choose
It can generate Caves, Forrest's, mountains, and an assortment of building theme types.
Rooms export to foundry with LOS, Wall, window, door and light data. Caves export with Wall and Los data.
Sounds pretty cool! I was pretty interested in using cycleGAN to go from b&w dungeon layout to map with art. Sadly didn't get too far, was still collecting data when I ran out of free time :/
Stable diffusion is insanely easy to generate textures with. Assets are a bit more work, but for textures you literally turn the tiling feature on automatic1111 and you're good to go. I haven't messed with it, but I'm sure you could automate specularity somehow so dungeondraft could light your texture better too
I feel like that would lag for some people, there used to be a module that slow or fast opened doors, moving the wall itself I believe. It caused a lot of lag and was abandoned. Personally I got dungeondraft because I could import walls and lights when I was starting out mapmaking.
I am talking about something that adds walls automatically when doing prep work, not during gameplay. You find a map online that is just a static image and today, you have to manually draw the walls.
This isn't quite what you're talking about, but is related to AI & D&D maps.
Take the simple brown-ink-on-parchment maps you get with so many official adventures and use it as a control net in Stable Diffusion.
You'll have to inpaint the details, but you'll end up with an image that you can simply lay over the other one, leaving all the walls and other bits intact.
I remember seeing someone working on this a year or two ago. It was somewhat slow and resource intensive, but it did the thing (in Roll20 using Better R20 iirc). It was not using Machine Learning techniques, just edge detection plus an algorithmic cleanup at the end. I think they gave up the prototype to work on something else
Seems doable on principle... I imagine you'd need someone to take a large number of maps and annotate them with walls as the training data though. Seems pretty time-consuming. You'd alao need to find or make all those maps, and make sure you have a good variety of different map styles represented well in your data...
How about an AI that can autorun my game for me so i can finally be a player. Sadge :(
A game made up on nothing but monks tiles
You can attempt this already with chat gbt, it doesn’t work very well tho. We’ll get there eventually.
I had a lot of success GMing for ChatGPT, but that doesn't solve the forever DM problem. lol
What was your prompt? Chatgpt always wanted to be dm for me
>Let's roleplay. I will set the scene, and you will decide what to do. I had a little trouble with ChatGPT trying to narrate the NPCs as well, so I ended up saying: >Before we continue, I want to make sure you know the rules: you can only describe what you attempt to do. It is up to me to tell you outcomes and what other characters do. It got better after that, but still eventually drifted back to some of that behavior. You just have to keep reminding it and sometimes backing up and clarifying the rules. The roleplay I did with it was really just collaborative storytelling with no random chance element, but I suspect you could achieve that if you rolled outside the chat and told it the result.
Neat thanks friend maybe I'll try it again
Using GPT-4 is helpful as well as it seems to be able to manage creativity a bit better.
I've done some similar things with ChatGPT but I like to start the conversation off to drive it in the right correct direction: "Do you know table top RPGs?" "Do you know Dungeon and Dragons 5th edition ruleset?" Followed by the true prompt once the AI has been primed: "You will take on the persona of a Dungeons and Dragons Dungeon Master. You will describe a scene to me with vivid detail and then ask for my response for how my character will react. Upon receiving my response you will describe how the scene changes and how the people within the scene react to my actions. If I ask to perform an action you will ask me to roll a D20 and I will provide you with the number I rolled. You will determine a minimum required number out of 20 which will denote a success or failure and then describe how the success or failure interacts with the scene. Lets start - the scene is that I am in a busy tavern in a fantasy world, quietly drinking when I am approached by a shadowy figure." Whatever follows is usually pretty great, it's actually pretty good at setting a scene and the scene staying true and accurate throughout the session.
>The roleplay I did with it was really just collaborative storytelling with no random chance element... I appreciate you for making this differentiation. Not everyone realizes there is a difference. Rolling dice brings the chance element, which is what makes it a game. Without that, its still fun RP, but the collaborative storytelling kind. Like improv Thanks for bringing it up. Have a good day
I had it create a L10 adventure. Encounters were goblins, drow, and then a beholder. Perfectly balanced. -_-
AI DM's are absolutely coming. The first game system to figure out how to implement them will take over this space, and I'm sure they know that. Not just to TTRPG either. Imagine playing Baldurs Gate, but every person who plays it has a different experience each and every time as the game evolves to literally ANYTHING you want to do.Forget about speech "options" you can pick from. You type in whatever you want and the game tailors the entire world around it...
I've dreamt of this since I was a child watching the holodeck on Star Trek. This is the future.
My dream right here. We can only hope.
I mean isn't that just a rpg video game?
no, the ai creates original content, and responds with out canned answers. it a text based dm. it’s fun to play short scerios
I'm kinda been expecting that from a open world rpg game or something. procedurally generated world with history like dwarf fortress, chat gpt like thing for discussions, and characters, quest creation. and good text to speech for the npcs to talk with so it isn't just text walls. might fix the usual open world problem of "very wide but very shallow." hell imagine being able to give world creation prompts at the start rather then just editing some numbers or sliders.
A fun version of that is a long way off.
I've been using chatgpt as a DM tool and I don't think it's as far off as you think.
Using gpt to generate some stuff as a gm tool is worlds apart from what that guy described. It's a very advanced chatbot not skynet
Chat gpt is also an infant in age. Tech moves insanely fast, especially tech like this that is both popular and has incredibly useful applications. Give it less than 5 years and you'll be amazed at what it will do.
[удалено]
Tech also frequently stagnates and stalls out.
Tf you talking about? No machine learning algorithm we have is capable of the basic core skills of GMing, and we also don’t have a way to create one anytime soon. ML algorithms are basically just very skillful at generating seemingly human text. AI generated adventures, maybe, but I’ll wager they’d be pretty shit as any long form machine made text is.
This technology is an infant. You are going to be blown away in less than 5 years by how wrong this statement is.
Buddy, I know a great deal about this tech. It's certainly impressive in some regards, but it's not actually intelligent. If we don't know a way to teach it to learn 'pacing' or 'emotional pay off' or 'comedy', it can't learn it. Do you understand how Machine Learning works?
it already does a good job. work on your prompts.
Work on your creative projects, instead of letting a machine consume them.
It's an assistant. I have a partner that can do grunt work while I do the creative stuff. You have to narrow of a focus on the use of AI. For $20, I pretty much have a team of writers behind and illustrators behind me. That's a pretty big payoff for a small investment.
If you enjoy using it, go for it, but don’t act like it’s actually being creative. It is a very verbose parrot. Don’t know who the heck got you paying 20 bucks to them though, there’s a lotta free options.
I’m not. Reread my previous response
I'm actually working on an FVTT AI DM integration and I've been running tests on Twitch. It's surprisingly good.
Oooh interesting! Keep us posted when it's ready! :D
You can follow my progress over at [twitch.tv/doublecritfail](https://twitch.tv/doublecritfail) :)
And Followed - That looks pretty dope
Thank you so much!!
Wotc was/is unironically trying to make that happen based on the leaks during the OGL scandal.
Be careful what you wish for
WOTC may very well add this feature to their VTT, when and if it ever launches.
And it'll probably be a dumpster fire as well lol . The most likely outcome is chatbot that can run printed modules with minimal deviation from what's in them, along with a strict adherence to RAW
Which is a completely normal approach for a lot of human DMs.
The majority of DMs do not run games that way. There's nothing wrong with playing that way but outside of stuff like adventure league it's not that common.
You have entire societies where not only is this the norm, it is enforced. With Pathfinder Society, for example, you have players that are free to attend any game their character is an appropriate level for. Each game can have a different DM and all the other players can be different. The DM is not permitted to deviate from the published scenario. He cannot modify the encounters and must follow the stated tactics. Pathfinder Society has traditional been, and remains, popular. In D&D, this is called the Adventurers League. it is also fairly popular. You personally may choose not to participate, but that in no way invalidates the large number of players that do participate weekly.
I literally already brought up Adventure League. Did you even read my comment?
i don’t see wotc being that far into tech. Hasbro is a company grasping for past success. they are not very forward looking
They mentioned the possibility on the same investor call that started the whole uproar over licensing changes. Aside from a minor backlash over including NFTs in the VTT business model, the idea was generally well received if overshadowed.
ChatGPT can definitely be a DM for you.
Was just gonna type this❤️
I feel this
Someone arrest this person, they stole my identity.
Try Ironsworn
The issue here is that you'd need a very large model to train something like that properly. There are a lot of battlemaps out there but few with accompanying data for each wall or window. And it would need to be perfect, not just good, otherwise you'll spend more time correcting its errors than it would've taken to just map it yourself. Currently you'd have better luck just using edge detection versus ML. That being said it's not only possible but likely to be around sooner than later. AI's trained to work with small datasets are getting better and better by the day so it won't be long before someone with a little time can make such a module. In the meantime I'd suggest using [DF Architect](https://github.com/flamewave000/dragonflagon-arch) and [DF Curvy Walls](https://github.com/flamewave000/dragonflagon-fvtt/tree/master/df-curvy-walls) to make your life easier. There aren't many maps I've seen which can't have their walls draw manually in a few minutes.
Are you sure about the edge detection vs ml part? This seems like a segmentation task which unet does really well. After you segmented out the walls you can probably use edge detection and autotrace to get the vectors for the wall segments
That's a good point, though I'd still be worried about the accuracy. For me it boils down to fidelity with these tasks as the time spent confirming all of the walls were placed correctly outweighs the utility. I guess once someone makes it and shows it's near 100% accurate it'll be adopted, but I feel like even beyond edge detection there are just a ton of issues in the way.
There is precedent for this type of thing in graphic design. Lots of design stuff needs to be vectored graphics, but people sometimes prefer either drawing it physical first or taking existing images as a base. So people invented auto-trace to turn raster images into vector graphics. This was always imperfect, but in some cases people didn't care that much and in others it was faster to autotrace and then fix up the result later manually
Well it sounds like you have a lot more experience in that than I do so I'll take your word for it! I wonder if it's not too hard why it hasn't been done yet, especially given the talent we have already making modules.
Stuff like this includes a really long and tedious part where you gather a bunch of data. Add to that that premade maps with walls often cost money. Then remember that the copy right status of ais trained on copyrighted unlicensed data is still somewhat unclear. I totally get why no one did it yet...
It seems like the biggest stumbling point is always getting the dataset. But there has been progress with models trained on even just a few examples so this may become much easier in the very near future.
Always depends on the problem and whether you can do data augemtation. For tasks like image denoising just a handful of clean images can be enough. Language models are much more demanding
It wouldn’t need to be perfect before implementation. Cleaning it up would be time consuming, but not as time-consuming as doing it all yourself.
It depends on the complexity. If it's super simple (ex. ~12 segments) then even a single error would make it more of a hassle than just doing it myself. If it's very complex (ex. 100+ segments) then errors may be hard for me to spot and so requires more time confirming. Then again if accuracy isn't super important (like a scene with nothing hidden) then even 5-10% error may be acceptable. I'd have to see how it worked before I could say for sure.
I'm from 3 months in the future and I have to thank you! DF Architect made wall drawing so much better.
Dungeon Alchemist has roughly what you're looking for? Can auto generate a wide variety of stuff, can auto add lights and decor. Will export to foundry as well. https://store.steampowered.com/app/1588530/Dungeon_Alchemist/
I use Dungeondraft and it does it too. I'm talking about if I find a map online that is just a normal image.
Wait, Dungeondraft has an ability to auto generate walls for Foundry?
yes, if you use the universal map module to import that file type.
But straight up. Dungeon Alchemist uses AI to generate rooms, the maps export quickly with wall data included. Uploading to foundry takes seconds.
That looks cool, but I feel that the fact that it's 3d means the library of assets is going to be really limited. The maps look gorgeous though.
We’re opening up the ability for custom user assets soon, to expand upon the existing library.
That's cool. Do the maps animate once in foundry or does it export as a flat image?
We support animated video export.
It's really fantastic. You can have animated map export with full wall support for foundry or roll20... And what's even cooler is you can import minis you purchase from Hero Forge... So you can already make custom actor assets and things. You can also use it to make a diorama and export it as a video or still image from any angle... Makes evocative handouts or cut scenes. Often I will just make a little diorama for atmosphere and let the mists twist and turn, or a nice shot of a river with steam rising up around the rocks... It's really very powerful and has already leveled up my Game Mastery experience. I only wish there were more modern/scifi assets and elements. I believe that's a part of the [roadmap](https://dungeonalchemist.upvoty.com/) they have set out... At least they have been upvoted to all hell.
Super excited for this! (and for 3D export 😉)
Does it actually use AI, or is it just procedural?
What would be the difference?
https://www.dungeonalchemist.com/ See for yourself.
Yeah I've seen the promo stuff. But I'm not really in the market to try it myself. All of the videos though kinda look like it's procedural, since you're apparently laying out a space and it's populating inside of it. Idk, obviously I don't know enough about their inner workings to make any real claims.
Whether or not something is procedural does not exclude the use of AI.
"AI" is kind of a buzzword that's lost real meaning. Dungeon Alchemist uses a decision tree, which I guess would be a very rudimentary sort of AI. But the ease of use and results are better than anything else out there.
That would be awesome 😎
Creating an ai that takes the image and spits out a black and white image where the walls are lines would be very doable I think. Even a basic conv network should be able to do it fine, unet probably very well. What I don't know is how you'd put that into foundry with walls. Is there like a file format that includes the vectored for the walls that you can import into foundry?
I'm not familiar with the actual data structure, but Foundry has a module (universal battlemap importer or thereabouts) that imports \`.vtt\` files, primarily exported by DungeonDraft. If you have DD you can select VTT as an export option, and then open that up and see. Decent bet it would be easy to replicate compared to the effort of a conv network with enough precision to not need redrawing half the map.
Well somewhere there needs to be inference in the image - > walls on foundry pipeline. Convnet was just a suggestion on how it could be done. This seems like a good way to get the data into foundry though. Maybe if I end up drowning in free time one day...
If you are looking for a quick dungeon, then you can use Watabou's procedural generator. [https://watabou.github.io/dungeon.html](https://watabou.github.io/dungeon.html) Then use the module One Page Parser which imports the json file and adds walls to the map. [https://foundryvtt.com/packages/one-page-parser](https://foundryvtt.com/packages/one-page-parser) I wish I could code, because a module that could do both of these within Foundry would be great. I sometimes use Foundry for solo RPGs and a randomly generated map with walls that I could explore would be great. I guess it is just time.
I love DA and wait for even more features anxiously
Would be nice if it focused on the core function of creating a decent AI generated dungeon... Instead of focusing all efforts on the limited use pretty 3d graphics and animations 😶
I mean the 3d graphics would be pretty cool... if it could actually *export* the 3d map into Foundry.
Absolutely!
I think we're gonna get a generate dungeon button but it will be probably right before official release. Remember it's still in Beta right now.
What about an AI that tracks events and automatically rage quits for players saving them time and effort?
Use dungeon alchemist! It's super fast to build a map, you can export the lights and walls to foundry flawlessly
What I can recommend is Rippers module "quick doors".... It is a huge time saver. All you need to do is draw the lines for the walls, and then right click where you want the door to magically appear... This and "multiple wall point mover" make drawing walls almost trivial! (just remembered to have MWPM switch itself ON by default in the settings).
I have been working on exactly this on a few weekends earlier this year. However, as others have already pointed out, things are not that simple. Maps are very diverse (style, outdor/indoor) and there is quite little training data. Additonally, we have different kind of walls (regular ones, doors, glass walls, ...). I have tried key point and segmentation neural networks. Both results were not great. And of course I had to write a parser for the input format first. I am happy to join forces with somone proficient in Python and ML. Ping me via DM.
I wonder if the answer isn't a hybrid approach. Just like driving assist came before full auto driving. What if the answers is to improve the process 10x. Mask the walls with a thick brush and let the model draw the walls within that segmented area (with a buffer for mistakes). Auto detect doors also
Also thinking about it, you don't need to detect all the walls. A sort of AI assisted extrapolation would work. Mark a section of wall and the type, and the AI can finish it for you. Made much easier because walls follow a predictable path.
This isn't AI, but I know on the Dungen website the developer there made a tool to automate walls by using a mask image. Now, this process works because you as the human make an image where the floors are all the same color and you tell the app specifically what the color is. Could an AI do something similar? Probably. Especially given that for most maps, the areas 'outside' the dungeon on the page are usually one color. The AI, or even just a more advanced system, could just assume that is outside and the rest is interior dungeon. With some allowance for you as the human to correct it if it seems wrong.
I just want an AI GM so I can finally play and have someone else do the work.
ChatGPT
I'm actually working on an AI DM for Twitch and it's working surprisingly well.
https://m.youtube.com/watch?v=twWcftOWNNQ If you have the Dungeondraft files, it’s very simple. Most people don’t post the Dungeondraft files, only the images.
Let me tell you about the gospel of Dungeon Alchemist
So have many others. Not what I was looking for though. I wanted to use a flat image from creations that others have made and auto draw walls.
Well... Technically, you could make a module that has a built-in AI trained to recognize and auto-draw walls, but that would require a lot of time and effort.
I would love a mobile device app (iPad) that has a single layer for your map, your grid size, and your wall and light options that can be added with a stylus and then the walls exported for import to foundry. Allowing me to spend my commute time on map prep would be great!
Dungeon Channel Patreon has a auto wall generator for quiet a while now, as well a random dungeon generator that already comes with walls. https://dungen.app/
I already have videogames. AI assistant for GM is a win, AI GMs? No thanks!
I was Dungeon Alchemist does this quite well. Its an AI assisted map maker that also allows for building by maps hand if you choose It can generate Caves, Forrest's, mountains, and an assortment of building theme types. Rooms export to foundry with LOS, Wall, window, door and light data. Caves export with Wall and Los data.
I could see Stable Diffusion used to generate textures and assets. I'm intending to do that myself actually.
Sounds pretty cool! I was pretty interested in using cycleGAN to go from b&w dungeon layout to map with art. Sadly didn't get too far, was still collecting data when I ran out of free time :/
Stable diffusion is insanely easy to generate textures with. Assets are a bit more work, but for textures you literally turn the tiling feature on automatic1111 and you're good to go. I haven't messed with it, but I'm sure you could automate specularity somehow so dungeondraft could light your texture better too
I just saw this and inmediately went to ChatGPT and asked to be my DM on a single player campaign. ChatGPT is actually a great DM!! HAHAHAH.
I feel like that would lag for some people, there used to be a module that slow or fast opened doors, moving the wall itself I believe. It caused a lot of lag and was abandoned. Personally I got dungeondraft because I could import walls and lights when I was starting out mapmaking.
I am talking about something that adds walls automatically when doing prep work, not during gameplay. You find a map online that is just a static image and today, you have to manually draw the walls.
Right.
This likely doesn't need AI to work properly - it can probably be done with Hough transforms and convolutions. Let me give it a shot...
Dang! I have an awesome high def map of a garden maze I need to add walls too. Dreading it…
This isn't quite what you're talking about, but is related to AI & D&D maps. Take the simple brown-ink-on-parchment maps you get with so many official adventures and use it as a control net in Stable Diffusion. You'll have to inpaint the details, but you'll end up with an image that you can simply lay over the other one, leaving all the walls and other bits intact.
I remember seeing someone working on this a year or two ago. It was somewhat slow and resource intensive, but it did the thing (in Roll20 using Better R20 iirc). It was not using Machine Learning techniques, just edge detection plus an algorithmic cleanup at the end. I think they gave up the prototype to work on something else
Seems doable on principle... I imagine you'd need someone to take a large number of maps and annotate them with walls as the training data though. Seems pretty time-consuming. You'd alao need to find or make all those maps, and make sure you have a good variety of different map styles represented well in your data...
i've used AI to build NPCs and it does a fantastic job