This video is an experiment in using AI to create a depth map that can be used to build a 3d model in Blender. I started out using the hugging face online models but eventually got into Stable diffusion using ComfyUI to combine various AI processes into one. I added a Marigold depth estimation node to extrude the surface as well as a Segment Anything model to mask out the windows for the roughness map. I am currently working on a blender addon to do the texture and modifier setup automatically
node setup in blender
ComfyUI setup (modified from https://openart.ai/workflows/jjg/pbr-material-prep-depth-maps/Iov7CK0dcdEStq9wA2W7)
Blender Addon script