A new tool called “Patchwork” was unveiled during a livestream demonstration led by Max Kreminski, head of Midjourney’s Storytelling Lab.
The demo was shared on both Discord and X (formerly Twitter) via Restream, showcasing Midjourney’s ambitions to explore AI-assisted storytelling.
Max Kreminski explained that Patchwork will function as a standalone app, requiring users to log in with their Midjourney accounts.
The research preview for the app will be accessible through a URL shared in the “updates” channel of the Midjourney Discord server.
To access Patchwork, users must link their Midjourney Discord accounts to their Google accounts, with detailed instructions provided on Midjourney’s X account.
The tool features an infinite white canvas, designed as a web-based interface, with a toolbox located on the left side of the screen. This toolbox includes options like:
- Character
- Event
- Faction
- Place
- Prop
- Random
Additionally, it offers tools for notes, images, portals, saving, and sharing. The “Save” function allows users to download a JSON file containing links to all Midjourney-generated images used within a canvas.
Each canvas is treated as its own distinct digital “world,” emphasizing the platform’s focus on interactive and immersive storytelling.
To navigate between different worlds within Patchwork, users can create a “portal,” represented by a small black circular button.
When it comes to generating a new world, users simply type a text prompt into the editor bar located at the top of the create screen. They can then choose from a selection of 10 distinct image styles to tailor the visual aesthetic of their new world.
This process generates a new whiteboard filled with various still image assets and text boxes, referred to as “scraps.”
These scraps serve as entities that can include input boxes, enabling the user to prompt additional images or settings that align with the initial world description.
Users can even generate entire new AI-driven character descriptions within these scraps, expanding the world’s narrative and visual elements.
During the demo livestream, the character name was automatically populated with Marcus “Dizzy” Gillespie, reminiscent of the renowned jazz musician.
When this description was dragged into a new character image creator box, it generated four new AI-created images, each visualizing the character based on the provided details.
By adding new character boxes, users can generate names, define characteristics, and craft motivations that drive conflicts for their story. These characters can then be connected with lines that visually represent their relationships.
Users can also write action sequences and scene descriptions to narrate the story, with each character being featured in multiple images. These images can be grouped together with a single option for easier management.
The platform allows users to “share” their boards with other Midjourney users, enabling collaborative creation, with multiple cursors moving across the same shared canvas in real-time.
A single world can accommodate dozens of users, potentially up to 100, though Kreminski mentioned that the experience may become chaotic with larger groups.
Kreminski shared that, at present, only logged-in users can view boards, but in the future, there may be an option for non-users to access them. He noted that tabletop roleplaying groups are already utilizing this feature to plan their campaigns.
Additionally, he revealed that Midjourney version 7 (V7) would introduce a setting aimed at maintaining character consistency across various images, ensuring continuity in visual representation.
Kreminski also revealed that Patchwork is powered by at least three large language models, including a unique, fine-tuned open-source model developed specifically for Midjourney.
The tool itself stands out as a powerful, complex, and somewhat overwhelming yet highly compelling platform for storyboarding.
Its versatility makes it ideal for a wide range of creators, from writers and filmmakers to game designers, comic book artists, and even live theater directors.
Looking ahead, Kreminski mentioned that there is a clear path toward expanding the platform’s capabilities, with the eventual goal of introducing fully immersive 3D virtual reality scenes. However, he cautioned that such advancements are still years away.
During the announcement livestream, Midjourney creator David Holz shared that the startup would soon roll out multiple personalization modes, giving users the ability to toggle between various personalized models.
Currently, Midjourney allows users to rate images and fine-tune the model to align with their preferences, but the new update will expand this with several versions for users to switch between.
Additionally, Holz revealed that Midjourney will introduce the option for users to upload and reference multiple images to guide their board generations.
Looking ahead, he confirmed that after Christmas, Midjourney would launch video models and a new version of the AI image generator (V7) with enhanced prompt understanding.
Holz also teased that Midjourney is working on three to four new hardware projects and plans to evolve into a full research lab, with announcements expected in the next six months.
Stories You May Like