website-enchun-mgr/apps/backend/scripts/migration

Webflow to Payload CMS Migration Script

Story 1.3: Content Migration Script

Overview

This script migrates content from Webflow CMS to Payload CMS. It supports:

• JSON export - If you have Webflow JSON export files
• HTML parsing - If you only have access to the public website HTML
• Manual entry - You can manually edit the JSON data file

Prerequisites

1. MongoDB must be running - The script connects to Payload CMS which requires MongoDB
2. Environment variables - Ensure .env file has PAYLOAD_SECRET and DATABASE_URI
3. Source data - Prepare your webflow-export.json file

Quick Start

# Navigate to backend directory
cd apps/backend

# Ensure MongoDB is running (if using local)
# Or the Payload CMS dev server:
pnpm dev

# In another terminal, run dry-run (preview mode, no changes)
pnpm migrate:dry

# Run actual migration
pnpm migrate

# Migrate specific collection
pnpm migrate:posts

# Show help
tsx scripts/migration/migrate.ts --help

Environment Setup

The script loads environment variables from:

• .env (project root)
• .env.enchun-cms-v2 (project root)
• apps/backend/.env

Required variables:

PAYLOAD_SECRET=your-secret-key
DATABASE_URI=mongodb://localhost:27017/your-db
R2_ACCOUNT_ID=your-r2-account
R2_ACCESS_KEY_ID=your-access-key
R2_SECRET_ACCESS_KEY=your-secret-key
R2_BUCKET=your-bucket-name

CLI Options

Option	Short	Description
--dry-run	-n	Run without making changes (preview mode)
--verbose	-v	Show detailed logging output
--force	-f	Overwrite existing items (skip deduplication)
--collection <name>	-c	Specific collection: categories
--source <path>	-s	Path to export file (default: ./data/webflow-export.json)
--batch-size <num>		Batch size for parallel processing (default: 5)
--help	-h	Show help message

Data Format

Create a webflow-export.json file in the apps/backend/data/ directory:

{
  "categories": [
    {
      "name": "分類名稱（中文）",
      "slug": "category-slug",
      "colorHex": "#0066cc"
    }
  ],
  "posts": [
    {
      "title": "文章標題",
      "slug": "post-slug",
      "content": "<p>HTML content...</p>",
      "publishedDate": "2024-01-15T10:00:00Z",
      "postCategory": "category-slug",
      "featuredImage": "https://example.com/image.jpg",
      "seoTitle": "SEO Title",
      "seoDescription": "SEO Description",
      "excerpt": "Article excerpt..."
    }
  ],
  "portfolio": [
    {
      "name": "作品名稱",
      "slug": "portfolio-slug",
      "websiteLink": "https://example.com",
      "previewImage": "https://example.com/preview.jpg",
      "description": "作品描述",
      "websiteType": "corporate",
      "tags": "tag1, tag2, tag3"
    }
  ]
}

Field Mappings

Categories

Webflow Field	Payload Field
name	title
slug	slug (preserved)
color-hex	textColor + backgroundColor

Posts

Webflow Field	Payload Field
title	title
slug	slug (preserved for SEO)
body	content (HTML → Lexical)
published-date	publishedAt
post-category	categories (relationship)
featured-image	heroImage (R2 upload)
seo-title	meta.title
seo-description	meta.description

Portfolio

Webflow Field	Payload Field
Name	title
Slug	slug
website-link	url
preview-image	image (R2 upload)
description	description
website-type	websiteType
tags	tags (array)

Migration Order

1. Categories (first - no dependencies)
2. Media images (independent)
3. Posts (depends on Categories and Media)
4. Portfolio (depends on Media)

Reports

After each migration, a report is generated in apps/backend/reports/:

• migration-YYYY-MM-DD.json - Machine-readable JSON
• migration-YYYY-MM-DD.md - Human-readable Markdown

Troubleshooting

Script fails to connect to Payload CMS

Ensure the Payload CMS server is running:

cd apps/backend
pnpm dev

Images not uploading

Check environment variables in .env:

• R2_ACCOUNT_ID
• R2_ACCESS_KEY_ID
• R2_SECRET_ACCESS_KEY
• R2_BUCKET_NAME

Duplicate entries

By default, the script skips existing items. Use --force to overwrite:

pnpm migrate --force

Module Structure

scripts/migration/
├── migrate.ts           # Main entry point
├── types.ts             # TypeScript interfaces
├── utils.ts             # Helper functions
├── transformers.ts      # Data transformation
├── mediaHandler.ts      # Image download/upload
├── deduplicator.ts      # Duplicate checking
├── reporter.ts          # Report generation
├── htmlParser.ts        # HTML parsing (no JSON)
└── README.md            # This file