Model: https://huggingface.co/OnomaAIResearch/Illustrious-XL-v2.0

Abstract

This report presents UniAnimate-DiT, an advanced project that leverages the cutting-edge and powerful capabilities of the open-source Wan2.1 model for consistent human image animation. Specifically, to preserve the robust generative capabilities of the original Wan2.1 model, we implement Low-Rank Adaptation (LoRA) technique to fine-tune a minimal set of parameters, significantly reducing training memory overhead. A lightweight pose encoder consisting of multiple stacked 3D convolutional layers is designed to encode motion information of driving poses. Furthermore, we adopt a simple concatenation operation to integrate the reference appearance into the model and incorporate the pose information of the reference image for enhanced pose alignment. Experimental results show that our approach achieves visually appearing and temporally consistent high-fidelity animations. Trained on 480p (832x480) videos, UniAnimate-DiT demonstrates strong generalization capabilities to seamless

It's been a while since I've updated my Stable Diffusion kit, and the technology moves so fast that I should probably figure out what new tech is out there.

Is most everyone still using AUTOMATIC's interface? Any cool plugins people are playing with? Good models?

What's the latest in video generation? I've seen a lot of animated images that seem to retain frame-to-frame adherence very well. Kling 1.6 is out there, but it doesn't appear to be free or local.

Release: https://github.com/mcmonkeyprojects/SwarmUI/releases/tag/0.9.6-Beta

Abstract

We present Liquid, an auto-regressive generation paradigm that seamlessly integrates visual comprehension and generation by tokenizing images into discrete codes and learning these code embeddings alongside text tokens within a shared feature space for both vision and language. Unlike previous multimodal large language model (MLLM), Liquid achieves this integration using a single large language model (LLM), eliminating the need for external pretrained visual embeddings such as CLIP. For the first time, Liquid uncovers a scaling law that performance drop unavoidably brought by the unified training of visual and language tasks diminishes as the model size increases. Furthermore, the unified token space enables visual generation and comprehension tasks to mutually enhance each other, effectively removing the typical interference seen in earlier models. We show that existing LLMs can serve as strong foundations for Liquid, saving 100x in training costs while outperforming Cham

Highlights for 2025-04-12

Last release was just over a week ago and here we are again with another update as a new high-end image model, HiDream-I1 jumped out and generated a lot of buzz!
There are quite a few other performance and quality-of-life improvements in this release and 40 commits, so please take a look at the full ChangeLog

ReadMe | ChangeLog | Docs | WiKi | Discord

Overview

Pusa introduces a paradigm shift in video diffusion modeling through frame-level noise control (thus it has thousands of timesteps, rather than one thousand of timesteps), departing from conventional approaches. This shift was first presented in our FVDM paper. Leveraging this architecture, Pusa seamlessly supports diverse video generation tasks (Text/Image/Video-to-Video) while maintaining exceptional motion fidelity and prompt adherence with our refined base model adaptations. Pusa-V0.5 represents an early preview based on Mochi1-Preview. We are open-sourcing this work to foster community collaboration, enhance methodologies, and expand capabilities.

Model: https://huggingface.co/RaphaelLiu/Pusa-V0.5

Code: https://github.com/Yaofang-Liu/Pusa-VidGen

Training Toolkit: https://github.com/Yaofang-Liu/Mochi-Full-Finetuner

Dataset: https://huggingface.co/datasets/RaphaelLiu/PusaV0.5_Trainin

Announcement: https://github.com/mit-han-lab/nunchaku/discussions/236

Some things are just meant for each other. Like Igorr and GenAI art.

Abstract

Recent advancements in Unet-based diffusion models, such as ControlNet and IP-Adapter, have introduced effective spatial and subject control mechanisms. However, the DiT (Diffusion Transformer) architecture still struggles with efficient and flexible control. To tackle this issue, we propose EasyControl, a novel framework designed to unify condition-guided diffusion transformers with high efficiency and flexibility. Our framework is built on three key innovations. First, we introduce a lightweight Condition Injection LoRA Module. This module processes conditional signals in isolation, acting as a plug-and-play solution. It avoids modifying the base model weights, ensuring compatibility with customized models and enabling the flexible injection of diverse conditions. Notably, this module also supports harmonious and robust zero-shot multi-condition generalization, even when trained only on single-condition data. Second, we propose a Position-Aware Training Paradigm. This ap