Documentation

Complete guide to using Open-Sora for video generation

Installation

Get started with Open-Sora by following these installation steps.

Prerequisites

  • Python 3.8 or higher
  • CUDA-capable GPU (recommended)
  • PyTorch installed

Install from Source

git clone https://github.com/hpcaitech/Open-Sora.git
cd Open-Sora
pip install -e .

Install Dependencies

pip install -r requirements.txt

Text-to-Video Generation

Generate videos from text prompts using Open-Sora.

Basic Usage

# Single GPU for 256px
torchrun --nproc_per_node 1 --standalone scripts/diffusion/inference.py \
    configs/diffusion/inference/256px.py \
    --prompt "raining, sea"

# Multi-GPU for 768px
torchrun --nproc_per_node 8 --standalone scripts/diffusion/inference.py \
    configs/diffusion/inference/768px.py \
    --prompt "raining, sea"

Parameters

  • --prompt: Text prompt describing the video
  • --aspect_ratio: Video aspect ratio (16:9, 9:16, 1:1, 2.39:1)
  • --num_frames: Number of frames (4k+1, less than 129)
  • --save-dir: Directory to save generated videos

Image-to-Video Generation

Generate videos from static images.

Single Image

torchrun --nproc_per_node 1 --standalone scripts/diffusion/inference.py \
    configs/diffusion/inference/256px.py \
    --cond_type i2v_head \
    --prompt "A plump pig wallows in a muddy pond..." \
    --ref assets/texts/i2v.png

Batch Processing

torchrun --nproc_per_node 1 --standalone scripts/diffusion/inference.py \
    configs/diffusion/inference/256px.py \
    --cond_type i2v_head \
    --dataset.data-path assets/texts/i2v.csv

Motion Score

Control the intensity of motion in generated videos.

Fixed Motion Score

torchrun --nproc_per_node 1 --standalone scripts/diffusion/inference.py \
    configs/diffusion/inference/t2i2v_256px.py \
    --prompt "raining, sea" \
    --motion-score 4

Dynamic Motion Score

Requires OpenAI API key:

export OPENAI_API_KEY=sk-xxxx
torchrun --nproc_per_node 1 --standalone scripts/diffusion/inference.py \
    configs/diffusion/inference/t2i2v_256px.py \
    --prompt "raining, sea" \
    --motion-score dynamic

Motion scores range from 1 (static) to 7 (high motion).

Prompt Refinement

Automatically refine prompts using ChatGPT for better results.

export OPENAI_API_KEY=sk-xxxx
torchrun --nproc_per_node 1 --standalone scripts/diffusion/inference.py \
    configs/diffusion/inference/t2i2v_256px.py \
    --prompt "raining, sea" \
    --refine-prompt True

Configuration

Customize generation settings for optimal results.

Reproducibility

torchrun --nproc_per_node 1 --standalone scripts/diffusion/inference.py \
    configs/diffusion/inference/t2i2v_256px.py \
    --prompt "raining, sea" \
    --sampling_option.seed 42 \
    --seed 42

Multiple Samples

--num-sample k  # Generate k samples for each prompt

API Reference

For detailed API documentation, please refer to the GitHub repository.

Command Line Arguments

  • --prompt: Text prompt for video generation
  • --cond_type: Condition type (i2v_head for image-to-video)
  • --ref: Reference image path
  • --motion-score: Motion intensity (1-7 or dynamic)
  • --refine-prompt: Enable prompt refinement
  • --seed: Random seed for reproducibility
  • --num-sample: Number of samples to generate