跪拜 Guibai
← Back to the summary

UnifiedJS Turns Any Text Format Into a Parse-Transform-Generate Pipeline

Preface

Hello everyone! I am a gust of autumn wind

In frontend development, "content format conversion" is a high-frequency requirement — writing a blog requires converting Markdown to HTML, building a backend requires converting rich text to plain text, and developing a component library requires converting MD documents to Vue/React components. However, traditional solutions always struggle with "tool stacking, chaotic logic, and poor reusability." The emergence of UnifiedJS is precisely to provide a "standardized, extensible" solution for frontend content processing.

This article will take you from basic understanding to practical implementation, helping you thoroughly master UnifiedJS — not just how to use it, but also understanding its principles.

1. What is UnifiedJS?

Many newcomers treat UnifiedJS as a "Markdown converter," but its positioning goes far beyond that — UnifiedJS is a "text processing framework." Its core function is to provide a standardized process of "Parse → Transform → Stringify" for texts in different formats (Markdown, HTML, JSX, etc.), achieving flexible extension through "pluginization."

1. Core Advantages: Why Choose UnifiedJS?

2. Applicable Scenarios and Boundaries

2. UnifiedJS Core Principles: Three-Stage Process + AST-Driven

All functions of UnifiedJS revolve around the "Parse → Transform → Stringify" three-stage process, and the core connecting this process is the "AST (Abstract Syntax Tree)" — which can be understood as a "structured description of text," allowing the program to precisely locate and modify content.

1. Three-Stage Process Breakdown

Taking "Markdown to HTML with highlighting" as an example, the complete process is as follows:

Process Stage Core Task Common Tools Example Role
Parse Convert raw Markdown text into an MD AST remark-parse Converts # Title **bold** into an AST with heading and strong nodes
Transform Traverse and modify the AST (e.g., add highlighting, remove formatting) remark-rehype (cross-format conversion), strip-markdown (remove formatting), rehype-highlight (code highlighting) Uses strip-markdown to convert heading nodes to plain text nodes, removing the # marker
Stringify Convert the modified AST into the target format remark-stringify (to plain text), rehype-html (to HTML) Converts the de-formatted AST into plain text without Markdown markers

2. AST: The "Structured Map" of Text

AST is the "common language" of UnifiedJS. Whether processing Markdown or HTML, everything is ultimately converted into an AST for manipulation. Taking an image in Markdown as an example, its AST structure (simplified version) is as follows:

{
  type: 'image', // Node type: image
  url: 'https://example.com/img.jpg', // Image link
  alt: 'Example Image', // Image alt attribute
  properties: { // Extended properties
    width: '400px'
  },
  position: { // Position info (for error location)
    start: { line: 5, column: 1 },
    end: { line: 5, column: 20 }
  }
}

By manipulating the properties of AST nodes (e.g., adding loading: 'lazy' to properties), you can easily implement the requirement of "adding lazy loading to all images." The core logic of strip-markdown is to uniformly convert formatted AST nodes like heading, strong, link into text nodes, thereby removing formatting.

3. UnifiedJS in Practice: From Environment Setup to Multi-Scenario Implementation

Next, through 3 practical cases, you will master the core usage of UnifiedJS. All code can be directly copied into your projects.

Case 1: Basic Introduction — Removing Markdown Formatting to Plain Text (strip-markdown Solution)

Requirement: Completely remove formatting markers from Markdown (like #, **, [](), | (tables)), retaining only the plain text content, without needing to manually handle edge cases (like code blocks, footnotes).

Key Notes

strip-markdown is an officially recommended plugin in the remark ecosystem, specifically designed for "converting formatted MD AST to plain text AST". It supports all standard Markdown syntax (including tables, code blocks, footnotes, and other scenarios hard to cover with custom plugins) and offers better maintainability (the plugin updates with the Markdown standard).

1. Environment Setup

# Install core dependencies (parse MD + strip formatting plugin + generate plain text)
npm install unified@10 remark-parse@10 strip-markdown@5 remark-stringify@10

2. Core Code

<script setup>
import { unified } from "unified";
import remarkParse from "remark-parse";
import stripMarkdown from "strip-markdown";
import remarkStringify from "remark-stringify";

// Build pipeline: Parse MD → Strip formatting → Generate plain text
const processor = unified()
  .use(remarkParse) // 1. Parse: Markdown → Formatted MD AST
  .use(stripMarkdown) // 2. Transform: Strip formatting (convert heading/strong etc. nodes to text nodes)
  .use(remarkStringify, {
    // 3. Stringify: Configure plain text output format (avoid extra blank lines/spaces)
    bullet: "", // Remove list item prefix (e.g., "-")
    emphasis: "", // Remove formatting from emphasized content (e.g., "**")
    fence: "", // Remove code block fences (```)
    listItemIndent: "1", // No indentation for list items
    newline: "\n", // Unify newline character to \n
    rule: "", // Remove horizontal rules
    ruleRepetition: 0,
    ruleSpaces: false,
  });

// Test content (including complex formats: headings, bold, links, lists, tables, code blocks)
const markdownText = `# My First Article
This is **bold content**, and a [Baidu link](https://baidu.com).

## List Content
- List item 1 (with parentheses)
- List item 2

## Table Example
| Name | Age |
|------|------|
| Zhang San | 25   |

## Code Block Example
\`\`\`js
// Simple code
console.log('hello');
\`\`\``;

// Execute conversion (synchronous processing, suitable for small text scenarios)
const result = processor.processSync(markdownText);
// Output plain text (automatically removes all formatting, retains content logic)
const resultText = String(result).trim();

console.log("### Conversion Result:", resultText);
</script>

<template>
  <div class="container">
    <h1 class="title">Unified.js Markdown Conversion Example</h1>
    
    <div class="comparison">
      <div class="panel">
        <h2 class="panel-title">Markdown Original</h2>
        <pre class="content">{{ markdownText }}</pre>
      </div>

      <div class="divider"></div>

      <div class="panel">
        <h2 class="panel-title">Plain Text (All Formatting Removed)</h2>
        <pre class="content">{{ resultText }}</pre>
      </div>
    </div>
  </div>
</template>

<style scoped>
.container {
  padding: 20px;
  max-width: 1400px;
  margin: 0 auto;
}

.title {
  text-align: center;
  color: #2c3e50;
  margin-bottom: 30px;
  font-size: 28px;
}

.comparison {
  display: flex;
  gap: 20px;
  align-items: stretch;
}

.panel {
  flex: 1;
  background: #f8f9fa;
  border-radius: 8px;
  padding: 20px;
  box-shadow: 0 2px 8px rgba(0, 0, 0, 0.1);
}

.panel-title {
  color: #42b983;
  margin: 0 0 15px 0;
  font-size: 20px;
  padding-bottom: 10px;
  border-bottom: 2px solid #42b983;
}

.content {
  background: white;
  padding: 15px;
  border-radius: 6px;
  border: 1px solid #e1e4e8;
  font-family: 'Courier New', monospace;
  font-size: 14px;
  line-height: 1.6;
  overflow-x: auto;
  white-space: pre-wrap;
  word-wrap: break-word;
  color: #333;
  margin: 0;
}

.divider {
  width: 2px;
  background: linear-gradient(to bottom, #42b983, #35495e);
  border-radius: 2px;
}

/* Responsive Design */
@media (max-width: 768px) {
  .comparison {
    flex-direction: column;
  }
  
  .divider {
    width: 100%;
    height: 2px;
  }
}
</style>

3. Effect Verification

Input the complex Markdown above, output plain text (all formatting removed, content logic clear):

image.png

Case 2: Practical Scenario — Markdown to HTML with Code Highlighting

Requirement: Automatically add syntax highlighting to code blocks in Markdown, ultimately outputting an HTML string (usable for web page rendering).

1. Install Dependencies

# Core dependencies + HTML conversion + code highlighting

npm install unified remark-parse remark-rehype rehype-html rehype-highlight 

2. Core Code

<script setup>
import { unified } from "unified";
import remarkParse from "remark-parse";
import remarkRehype from "remark-rehype";
import rehypeHighlight from "rehype-highlight";
import { toHtml } from "hast-util-to-html";
// Import highlight.js styles
import "highlight.js/styles/github-dark.css";

// Build pipeline: MD → MD AST → HTML AST → Highlighted HTML
const processor = unified()
  .use(remarkParse) // Parse: MD → MD AST
  .use(remarkRehype, { allowDangerousHtml: false }) // Transform: MD AST → HTML AST (Key: cross-format AST conversion)
  .use(rehypeHighlight); // Transform: Add highlight classes to code blocks in HTML AST

// Test MD with code blocks
const mdWithCode = `# Code Highlighting Test

Below is JavaScript code:

\`\`\`js
function add(a, b) {
  return a + b;
}
console.log(add(1, 2)); // Output 3
\`\`\`

## Python Code Example

\`\`\`python
def multiply(x, y):
    return x * y

result = multiply(5, 3)
print(f"Result is: {result}")
\`\`\`

## Plain Text

This is **bold** text, and a [link](https://example.com).`;


// 1. First convert MD to HTML AST
const tree = processor.runSync(processor.parse(mdWithCode));
// 2. Use toHtml to convert AST to HTML string
const htmlResult = toHtml(tree, { allowDangerousHtml: false });

console.log("### Converted HTML:", htmlResult);
</script>

<template>
  <div class="container">
    <h1 class="title">Unified.js Markdown to HTML (with Code Highlighting)</h1>
    
    <div class="comparison">
      <div class="panel">
        <h2 class="panel-title">Markdown Original</h2>
        <pre class="content">{{ mdWithCode }}</pre>
      </div>

      <div class="divider"></div>

      <div class="panel">
        <h2 class="panel-title">Generated HTML Code</h2>
        <div class="config-info">
          remarkParse → remarkRehype → rehypeHighlight → rehypeStringify
        </div>
        <pre class="content html-code">{{ htmlResult }}</pre>
      </div>

      <div class="divider"></div>

      <div class="panel">
        <h2 class="panel-title">Rendered Effect (with Highlighting)</h2>
        <div class="content rendered" v-html="htmlResult"></div>
      </div>
    </div>
  </div>
</template>

3. Effect Description

image.png

Case 3: — Implementing Markdown Preview in a Vue Component

Requirement: Encapsulate a MarkdownPreview component in a Vue 3 project, supporting the input of Markdown content and real-time rendering into highlighted HTML.

1. Environment Setup (Vue Project)

npm install unified remark-parse remark-stringify strip-markdown remark-rehype rehype-highlight hast-util-to-html highlight.js

2. Encapsulate Vue Component (src/components/MarkdownPreview.vue)

<template>
  <div class="md-preview" v-html="renderedHtml"></div>
</template>

<script setup>
import { ref, watch, onMounted } from "vue";
import { unified } from "unified";
import remarkParse from "remark-parse";
import remarkRehype from "remark-rehype";
import rehypeHighlight from "rehype-highlight";
import { toHtml } from "hast-util-to-html";

const props = defineProps({
  content: {
    type: String,
    required: true,
    default: "# Please input Markdown content",
  },
});

// Store the converted HTML string
const renderedHtml = ref("");

// 1. Build pipeline (reuse logic, avoid repeated initialization)
const createProcessor = () => {
  return unified()
    .use(remarkParse) // MD → MD AST
    .use(remarkRehype, { allowDangerousHtml: false }) // MD AST → HTML AST
    .use(rehypeHighlight); // Code highlighting (add class to HTML AST)
};

// 2. Core function for MD to HTML conversion (using synchronous method)
const convertMdToHtml = (mdContent) => {
  try {
    const processor = createProcessor();
    // Synchronous processing
    const tree = processor.runSync(processor.parse(mdContent));
    renderedHtml.value = toHtml(tree, { allowDangerousHtml: false });
  } catch (error) {
    // Error handling (e.g., MD syntax error)
    renderedHtml.value = `<div class="error">Markdown parsing error: ${error.message}</div>`;
  }
};

// 3. Watch content changes, update preview in real-time (execute once on initial render)
watch(
  () => props.content,
  (newContent) => convertMdToHtml(newContent),
  { immediate: true }
);

// 4. Import code highlighting styles (load in onMounted to reduce initial bundle size)
onMounted(() => {
  // Can be replaced with other themes (e.g., atom-one-dark.css, monokai.css, github-dark.css)
  import("highlight.js/styles/github.css");
});
</script>

3. Using the Component (src/views/HomeView.vue)

<template>
  <div class="demo-page">
    <h1 class="page-title">Vue + UnifiedJS Real-time Markdown Preview</h1>
    
    <div class="editor-container">
      <!-- Left Editor -->
      <div class="editor-panel">
        <h2 class="panel-title">
          <span class="icon">✏️</span>
          Markdown Editor
        </h2>
        <textarea
          v-model="markdownContent"
          class="markdown-editor"
          placeholder="Enter Markdown content here..."
        ></textarea>
      </div>

      <div class="divider"></div>

      <!-- Right Preview -->
      <div class="preview-panel">
        <h2 class="panel-title">
          <span class="icon">👁️</span>
          Real-time Preview
        </h2>
        <div class="preview-wrapper">
          <MarkdownPreview :content="markdownContent" />
        </div>
      </div>
    </div>

    <!-- Example Buttons -->
    <div class="examples">
      <h3>Quick Examples:</h3>
      <button
        v-for="example in examples"
        :key="example.name"
        @click="loadExample(example.content)"
        class="example-btn"
      >
        {{ example.name }}
      </button>
    </div>
  </div>
</template>

<script setup>
import { ref } from 'vue'
import MarkdownPreview from '../../components/MarkdownPreview.vue'

// Initial Markdown content
const markdownContent = ref(`# Vue + UnifiedJS in Practice

This is a Markdown preview example **with code highlighting**.

## 1. Features

- ✅ **Real-time Preview**: Render as you type
- ✅ **Code Highlighting**: Supports multiple programming languages
- ✅ **Component-based**: Reusable Vue component
- ✅ **Security**: Prevents XSS attacks

## 2. Code Examples

### JavaScript Example

\`\`\`js
function fibonacci(n) {
  if (n <= 1) return n;
  return fibonacci(n - 1) + fibonacci(n - 2);
}

console.log(fibonacci(10)); // Output: 55
\`\`\`

### Python Example

\`\`\`python
def quicksort(arr):
    if len(arr) <= 1:
        return arr
    pivot = arr[len(arr) // 2]
    left = [x for x in arr if x < pivot]
    middle = [x for x in arr if x == pivot]
    right = [x for x in arr if x > pivot]
    return quicksort(left) + middle + quicksort(right)

print(quicksort([3, 6, 8, 10, 1, 2, 1]))
\`\`\`

## 3. Other Elements

### Blockquote

> This is a blockquote.
> It can be used to display important information or quote others.

### Table

| Language | Type | Feature |
|------|------|------|
| JavaScript | Dynamic | Flexible, widely used |
| Python | Dynamic | Concise, easy to learn |
| TypeScript | Static | Type-safe |

### Links and Images

Visit the [Unified official website](https://unifiedjs.com/) for more information.

### Inline Code

Use \`npm install unified\` to install the core package.

---

**Tip**: Modify the content in the left editor, and the right side will update the preview in real-time!
`)

// Preset examples
const examples = [
  {
    name: 'Basic Example',
    content: `# Heading Example

## Second-level Heading

This is a paragraph of normal text, containing **bold** and *italic*.

- List item 1
- List item 2
- List item 3

> This is a blockquote`
  },
  {
    name: 'Code Example',
    content: `# Code Highlighting Example

## JavaScript

\`\`\`js
const greeting = (name) => {
  return \`Hello, \${name}!\`;
}
console.log(greeting('World'));
\`\`\`

## Python

\`\`\`python
def greet(name):
    return f"Hello, {name}!"
print(greet("World"))
\`\`\``
  },
  {
    name: 'Table Example',
    content: `# Table Example

## Fruit Price List

| Fruit | Price (Yuan/Jin) | Origin |
|------|------------|------|
| Apple | 5 | Shandong |
| Banana | 3 | Hainan |
| Orange | 4 | Jiangxi |

## Description

Tables support automatic alignment and formatting.`
  }
]

// Load example
const loadExample = (content) => {
  markdownContent.value = content
}
</script>

4. Demo Effect

1.gif

4. UnifiedJS Advanced: Performance Optimization and Ecosystem Expansion

1. Performance Optimization: Handling Large Files and High-Frequency Conversions

1.1 Caching Pipeline Instances (Avoiding Repeated Initialization)

When Markdown needs to be converted frequently (e.g., real-time preview in an editor), repeatedly creating the processor wastes performance. This can be optimized by caching the instance using a "singleton pattern":

// Optimization in MarkdownPreview.vue
let cachedProcessor = null; // Cache processor instance

const getProcessor = () => {
  if (!cachedProcessor) {
    cachedProcessor = unified()
      .use(remarkParse)
      .use(remarkRehype)
      .use(rehypeHighlight)
      .use(rehypeHtml);
  }
  return cachedProcessor;
};

// Call cached instance in conversion function
const convertMdToHtml = async (mdContent) => {
  try {
    const processor = getProcessor(); // Reuse cached instance
    const result = await processor.process(mdContent);
    renderedHtml.value = String(result);
  } catch (error) {
    renderedHtml.value = `<div class="error">Parsing error: ${error.message}</div>`;
  }
};

Optimization Effect: For high-frequency conversions (e.g., triggered 5 times per second), performance improves by about 40% (reducing initialization overhead).

1.2 Chunked Processing for Large Files (Avoiding Blocking)

When processing Markdown over 100,000 words (like long documents), synchronous conversion blocks the main thread, causing page lag. This can be optimized through "chunked processing + Web Worker":

// 1. Create Web Worker (src/workers/mdProcessor.js)
import { unified } from 'unified';
import remarkParse from 'remark-parse';
import remarkRehype from 'remark-rehype';
import rehypeHtml from 'rehype-html';

// Listen for messages from the main thread
self.onmessage = async (e) => {
  const { mdContent } = e.data;
  try {
    // Chunked processing: Split by paragraphs (avoids parsing large text all at once)
    const mdChunks = mdContent.split('\n\n'); // Split paragraphs by blank lines
    const htmlChunks = [];

    const processor = unified()
      .use(remarkParse)
      .use(remarkRehype)
      .use(rehypeHtml);

    // Parse paragraph by paragraph, send progress to main thread after each chunk
    for (let i = 0; i < mdChunks.length; i++) {
      if (mdChunks[i].trim()) {
        const result = await processor.process(mdChunks[i]);
        htmlChunks.push(String(result));
        // Send progress (e.g., "30% complete")
        self.postMessage({
          type: 'progress',
          progress: Math.round((i + 1) / mdChunks.length * 100)
        });
      }
    }

    // Processing complete, send final HTML
    self.postMessage({
      type: 'complete',
      html: htmlChunks.join('\n\n')
    });
  } catch (error) {
    self.postMessage({ type: 'error', message: error.message });
  }
};

// 2. Use Web Worker in Vue component
// Modify convertMdToHtml function in MarkdownPreview.vue
const convertMdToHtml = async (mdContent) => {
  // Only use Web Worker for large files (>10000 characters)
  if (mdContent.length > 10000) {
    // Create Worker instance
    const worker = new Worker(new URL('@/workers/mdProcessor.js', import.meta.url), {
      type: 'module'
    });

    // Listen for Worker messages
    worker.onmessage = (e) => {
      if (e.data.type === 'progress') {
        // Show progress (e.g., progress bar)
        console.log(`Parsing progress: ${e.data.progress}%`);
      } else if (e.data.type === 'complete') {
        renderedHtml.value = e.data.html;
        worker.terminate(); // Close Worker
      } else if (e.data.type === 'error') {
        renderedHtml.value = `<div class="error">Parsing error: ${e.data.message}</div>`;
        worker.terminate();
      }
    };

    // Send Markdown content to Worker
    worker.postMessage({ mdContent });
  } else {
    // Still use synchronous processing for small files (more efficient)
    const processor = getProcessor();
    const result = await processor.process(mdContent);
    renderedHtml.value = String(result);
  }
};

2. Ecosystem Expansion: Common UnifiedJS Plugins and Scenarios

The core advantage of UnifiedJS is its "plugin ecosystem." Here are common plugin combinations for different scenarios, ready for direct reuse:

2.1 Documentation Generation Scenario (e.g., Component Library Docs)

Requirement: Markdown to HTML + Auto-generate TOC + Code Highlighting + Image Lazy Loading

const processor = unified()
  .use(remarkParse) // MD → AST
  .use(remark-toc, { heading: 'Table of Contents' }) // Auto-generate TOC (matches "Table of Contents" heading)
  .use(remark-add-img-lazy) // Image lazy loading
  .use(remarkRehype) // MD AST → HTML AST
  .use(rehype-highlight) // Code highlighting
  .use(rehype-slug) // Add IDs to headings (for TOC anchor jumps)
  .use(rehype-autolink-headings) // Add anchor links to headings (click to jump to itself)
  .use(rehype-html); // HTML AST → HTML string

2.2 WeChat Official Account Formatting Scenario

Requirement: Markdown to HTML conforming to WeChat Official Account style (adapted for mobile)

const processor = unified()
  .use(remarkParse)
  .use(remarkRehype)
  .use(rehype-highlight)
  .use(rehype-weixin-style) // WeChat style adaptation (e.g., font, line height)
  .use(rehype-img-process, { // Image processing: convert to WeChat temporary link/add watermark
    watermark: 'My Official Account', // Add watermark text
    maxWidth: 677 // Maximum width for WeChat images
  })
  .use(rehype-html);

5. Summary

UnifiedJS is not "another Markdown tool," but a "standardized framework for frontend content processing" — through its "AST-driven + plugin-based" design, it solves the pain points of traditional tools like "format fragmentation and difficult logic reuse," allowing developers to "process all text formats with a single workflow."

With the development of frontend documentation tools, low-code platforms, and content management systems, the UnifiedJS ecosystem will continue to improve: on one hand, the official team will optimize the "newcomer experience" (e.g., simplifying configuration, adding scenario-based templates); on the other hand, the community will produce more plugins for vertical scenarios (e.g., AI content processing, multi-language translation).

For developers, mastering UnifiedJS not only improves the efficiency of "content processing" but also fosters a mindset of "structurally processing text" — a mindset equally applicable in scenarios like editor development and data cleaning, making it an important skill for frontend engineers.

If you encounter problems in practice, feel free to discuss them in the comments section, or get help through the UnifiedJS GitHub community (UnifiedJS GitHub).

Comments

Top 1 of 2 from juejin.cn, machine-translated. The original thread is authoritative.

蓝莓Zz

Great!!!

秋天的一阵风

[Rose]