AI Runner 3.0.0 Development Preview: Draw and generate

Summary

In this demonstration, the presenter introduces a new feature in an upcoming version of a creative application that utilizes two canvases: one for drawing and the other for rendering images. Utilizing the Stable Diffusion model, the presenter showcases how to generate a realistic image of a majestic mountain with a river running from it, simply by typing a prompt and adjusting the generation steps for speed. The demonstration highlights the interactive process of drawing on one canvas to enhance the generated image on the other, planning future improvements to manage multiple requests efficiently, and showing the potential for users to refine their creations through iterative drawing and prompt adjustments.

Takeaways

• 🖌️ The software features two canvases: one for drawing (left) and one for rendering images (right).
• 📷 The right canvas is used to render professional photographs, such as majestic mountains on sunny days with rivers.
• 👩‍🔬 The software utilizes 'Stable Diffusion' model, known for generating realistic photos, applicable to various subjects beyond just people.
• ⚙️ The demonstration involves lowering the generation steps to 15 to expedite the rendering process.
• 🔁 Upon releasing the mouse button after drawing, the software submits a new request for image-to-image conversion, using the left canvas as the base.
• 🎨 Users can draw different elements (e.g., fields, sky, mountains) which then get integrated into the rendered image on the right canvas.
• 📈 Each release of the button sends a new request, potentially leading to a queue of requests needing backend processing.
• 💡 A planned enhancement is to process only the latest request from the drawing canvas to improve efficiency and reduce load.
• 🚀 Performance may vary based on hardware, with faster video cards providing quicker results.
• 🎨🖼️ The tool allows iterative refining by adding elements such as snow or rivers to the drawing, which then reflect in the rendered image.
• ☁️ Users can add additional features like clouds to improve the composition and realism of the rendered image.

Q & A

What are the two canvases shown in the video for?

The left canvas is for drawing and the right canvas is for rendering the AI-generated image.

What AI model is being used to generate the images?

The video shows using Stable Diffusion model to generate the images.

How does the presenter update the generated image?

By drawing on the left canvas and releasing the mouse button, a new request is made to update the image using the drawing as a base.

What enhancement is planned for multiple requests in queue?

The presenter plans to only use the latest request instead of processing all requests in queue to speed things up.

How could the generated images be improved?

By working on the prompt wording or adding more details to the drawing on the left canvas.

Why does the presenter lower the steps to 15 initially?

To make the image generation process quicker for the demo.

What elements does the presenter add to the drawing?

A field, blue background for the sky, details on the mountain, snow, a river, and some clouds.

Why does the presenter say the generated image looks terrible?

Because the image quality is not refined and needs further improvements to prompt wording and drawing.

How are multiple requests handled when drawing?

A new request is made each time a drawing change is made. This can result in a queue of requests.

What format is used for the Q&A output?

The output is returned in JSON format containing a list of question and answer objects.