UnifiedJS Turns Any Text Format Into a Parse-Transform-Generate Pipeline
Preface
Hello everyone! I am a gust of autumn wind
In frontend development, "content format conversion" is a high-frequency requirement — writing a blog requires converting Markdown to HTML, building a backend requires converting rich text to plain text, and developing a component library requires converting MD documents to Vue/React components. However, traditional solutions always struggle with "tool stacking, chaotic logic, and poor reusability." The emergence of UnifiedJS is precisely to provide a "standardized, extensible" solution for frontend content processing.
This article will take you from basic understanding to practical implementation, helping you thoroughly master UnifiedJS — not just how to use it, but also understanding its principles.
1. What is UnifiedJS?
Many newcomers treat UnifiedJS as a "Markdown converter," but its positioning goes far beyond that — UnifiedJS is a "text processing framework." Its core function is to provide a standardized process of "Parse → Transform → Stringify" for texts in different formats (Markdown, HTML, JSX, etc.), achieving flexible extension through "pluginization."
1. Core Advantages: Why Choose UnifiedJS?
- Cross-format Compatibility: One set of logic handles multiple text formats, such as using the same plugin mechanism to modify content in Markdown and HTML;
- Rich Plugin Ecosystem: The official and community provide hundreds of plugins (e.g.,
remark-tocfor automatic table of contents generation,rehype-highlightfor code highlighting,strip-markdownfor removing formatting), ready to use out of the box; - Highly Customizable: Supports custom plugins to modify the text's AST (Abstract Syntax Tree) to meet personalized needs (e.g., adding lazy loading to images, filtering sensitive words);
- Engineering-Friendly: Compatible with build tools like Vite, Webpack, Rollup, and can be seamlessly integrated into projects.
2. Applicable Scenarios and Boundaries
- Suitable Scenarios: Documentation tools (like VuePress), component library docs, CMS systems, content editors, and other scenarios requiring flexible text processing;
- Unsuitable Scenarios: When only simple conversion is needed (like Markdown to HTML for a static page), using lightweight tools like
markedis more efficient.
2. UnifiedJS Core Principles: Three-Stage Process + AST-Driven
All functions of UnifiedJS revolve around the "Parse → Transform → Stringify" three-stage process, and the core connecting this process is the "AST (Abstract Syntax Tree)" — which can be understood as a "structured description of text," allowing the program to precisely locate and modify content.
1. Three-Stage Process Breakdown
Taking "Markdown to HTML with highlighting" as an example, the complete process is as follows:
| Process Stage | Core Task | Common Tools | Example Role |
|---|---|---|---|
| Parse | Convert raw Markdown text into an MD AST | remark-parse |
Converts # Title **bold** into an AST with heading and strong nodes |
| Transform | Traverse and modify the AST (e.g., add highlighting, remove formatting) | remark-rehype (cross-format conversion), strip-markdown (remove formatting), rehype-highlight (code highlighting) |
Uses strip-markdown to convert heading nodes to plain text nodes, removing the # marker |
| Stringify | Convert the modified AST into the target format | remark-stringify (to plain text), rehype-html (to HTML) |
Converts the de-formatted AST into plain text without Markdown markers |
2. AST: The "Structured Map" of Text
AST is the "common language" of UnifiedJS. Whether processing Markdown or HTML, everything is ultimately converted into an AST for manipulation. Taking an image in Markdown as an example, its AST structure (simplified version) is as follows:
{
type: 'image', // Node type: image
url: 'https://example.com/img.jpg', // Image link
alt: 'Example Image', // Image alt attribute
properties: { // Extended properties
width: '400px'
},
position: { // Position info (for error location)
start: { line: 5, column: 1 },
end: { line: 5, column: 20 }
}
}
By manipulating the properties of AST nodes (e.g., adding loading: 'lazy' to properties), you can easily implement the requirement of "adding lazy loading to all images." The core logic of strip-markdown is to uniformly convert formatted AST nodes like heading, strong, link into text nodes, thereby removing formatting.
3. UnifiedJS in Practice: From Environment Setup to Multi-Scenario Implementation
Next, through 3 practical cases, you will master the core usage of UnifiedJS. All code can be directly copied into your projects.
Case 1: Basic Introduction — Removing Markdown Formatting to Plain Text (strip-markdown Solution)
Requirement: Completely remove formatting markers from Markdown (like #, **, [](), | (tables)), retaining only the plain text content, without needing to manually handle edge cases (like code blocks, footnotes).
Key Notes
strip-markdown is an officially recommended plugin in the remark ecosystem, specifically designed for "converting formatted MD AST to plain text AST". It supports all standard Markdown syntax (including tables, code blocks, footnotes, and other scenarios hard to cover with custom plugins) and offers better maintainability (the plugin updates with the Markdown standard).
1. Environment Setup
# Install core dependencies (parse MD + strip formatting plugin + generate plain text)
npm install unified@10 remark-parse@10 strip-markdown@5 remark-stringify@10
2. Core Code
<script setup>
import { unified } from "unified";
import remarkParse from "remark-parse";
import stripMarkdown from "strip-markdown";
import remarkStringify from "remark-stringify";
// Build pipeline: Parse MD → Strip formatting → Generate plain text
const processor = unified()
.use(remarkParse) // 1. Parse: Markdown → Formatted MD AST
.use(stripMarkdown) // 2. Transform: Strip formatting (convert heading/strong etc. nodes to text nodes)
.use(remarkStringify, {
// 3. Stringify: Configure plain text output format (avoid extra blank lines/spaces)
bullet: "", // Remove list item prefix (e.g., "-")
emphasis: "", // Remove formatting from emphasized content (e.g., "**")
fence: "", // Remove code block fences (```)
listItemIndent: "1", // No indentation for list items
newline: "\n", // Unify newline character to \n
rule: "", // Remove horizontal rules
ruleRepetition: 0,
ruleSpaces: false,
});
// Test content (including complex formats: headings, bold, links, lists, tables, code blocks)
const markdownText = `# My First Article
This is **bold content**, and a [Baidu link](https://baidu.com).
## List Content
- List item 1 (with parentheses)
- List item 2
## Table Example
| Name | Age |
|------|------|
| Zhang San | 25 |
## Code Block Example
\`\`\`js
// Simple code
console.log('hello');
\`\`\``;
// Execute conversion (synchronous processing, suitable for small text scenarios)
const result = processor.processSync(markdownText);
// Output plain text (automatically removes all formatting, retains content logic)
const resultText = String(result).trim();
console.log("### Conversion Result:", resultText);
</script>
<template>
<div class="container">
<h1 class="title">Unified.js Markdown Conversion Example</h1>
<div class="comparison">
<div class="panel">
<h2 class="panel-title">Markdown Original</h2>
<pre class="content">{{ markdownText }}</pre>
</div>
<div class="divider"></div>
<div class="panel">
<h2 class="panel-title">Plain Text (All Formatting Removed)</h2>
<pre class="content">{{ resultText }}</pre>
</div>
</div>
</div>
</template>
<style scoped>
.container {
padding: 20px;
max-width: 1400px;
margin: 0 auto;
}
.title {
text-align: center;
color: #2c3e50;
margin-bottom: 30px;
font-size: 28px;
}
.comparison {
display: flex;
gap: 20px;
align-items: stretch;
}
.panel {
flex: 1;
background: #f8f9fa;
border-radius: 8px;
padding: 20px;
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.1);
}
.panel-title {
color: #42b983;
margin: 0 0 15px 0;
font-size: 20px;
padding-bottom: 10px;
border-bottom: 2px solid #42b983;
}
.content {
background: white;
padding: 15px;
border-radius: 6px;
border: 1px solid #e1e4e8;
font-family: 'Courier New', monospace;
font-size: 14px;
line-height: 1.6;
overflow-x: auto;
white-space: pre-wrap;
word-wrap: break-word;
color: #333;
margin: 0;
}
.divider {
width: 2px;
background: linear-gradient(to bottom, #42b983, #35495e);
border-radius: 2px;
}
/* Responsive Design */
@media (max-width: 768px) {
.comparison {
flex-direction: column;
}
.divider {
width: 100%;
height: 2px;
}
}
</style>
3. Effect Verification
Input the complex Markdown above, output plain text (all formatting removed, content logic clear):
Case 2: Practical Scenario — Markdown to HTML with Code Highlighting
Requirement: Automatically add syntax highlighting to code blocks in Markdown, ultimately outputting an HTML string (usable for web page rendering).
1. Install Dependencies
# Core dependencies + HTML conversion + code highlighting
npm install unified remark-parse remark-rehype rehype-html rehype-highlight
2. Core Code
<script setup>
import { unified } from "unified";
import remarkParse from "remark-parse";
import remarkRehype from "remark-rehype";
import rehypeHighlight from "rehype-highlight";
import { toHtml } from "hast-util-to-html";
// Import highlight.js styles
import "highlight.js/styles/github-dark.css";
// Build pipeline: MD → MD AST → HTML AST → Highlighted HTML
const processor = unified()
.use(remarkParse) // Parse: MD → MD AST
.use(remarkRehype, { allowDangerousHtml: false }) // Transform: MD AST → HTML AST (Key: cross-format AST conversion)
.use(rehypeHighlight); // Transform: Add highlight classes to code blocks in HTML AST
// Test MD with code blocks
const mdWithCode = `# Code Highlighting Test
Below is JavaScript code:
\`\`\`js
function add(a, b) {
return a + b;
}
console.log(add(1, 2)); // Output 3
\`\`\`
## Python Code Example
\`\`\`python
def multiply(x, y):
return x * y
result = multiply(5, 3)
print(f"Result is: {result}")
\`\`\`
## Plain Text
This is **bold** text, and a [link](https://example.com).`;
// 1. First convert MD to HTML AST
const tree = processor.runSync(processor.parse(mdWithCode));
// 2. Use toHtml to convert AST to HTML string
const htmlResult = toHtml(tree, { allowDangerousHtml: false });
console.log("### Converted HTML:", htmlResult);
</script>
<template>
<div class="container">
<h1 class="title">Unified.js Markdown to HTML (with Code Highlighting)</h1>
<div class="comparison">
<div class="panel">
<h2 class="panel-title">Markdown Original</h2>
<pre class="content">{{ mdWithCode }}</pre>
</div>
<div class="divider"></div>
<div class="panel">
<h2 class="panel-title">Generated HTML Code</h2>
<div class="config-info">
remarkParse → remarkRehype → rehypeHighlight → rehypeStringify
</div>
<pre class="content html-code">{{ htmlResult }}</pre>
</div>
<div class="divider"></div>
<div class="panel">
<h2 class="panel-title">Rendered Effect (with Highlighting)</h2>
<div class="content rendered" v-html="htmlResult"></div>
</div>
</div>
</div>
</template>
3. Effect Description
Case 3: — Implementing Markdown Preview in a Vue Component
Requirement: Encapsulate a MarkdownPreview component in a Vue 3 project, supporting the input of Markdown content and real-time rendering into highlighted HTML.
1. Environment Setup (Vue Project)
npm install unified remark-parse remark-stringify strip-markdown remark-rehype rehype-highlight hast-util-to-html highlight.js
2. Encapsulate Vue Component (src/components/MarkdownPreview.vue)
<template>
<div class="md-preview" v-html="renderedHtml"></div>
</template>
<script setup>
import { ref, watch, onMounted } from "vue";
import { unified } from "unified";
import remarkParse from "remark-parse";
import remarkRehype from "remark-rehype";
import rehypeHighlight from "rehype-highlight";
import { toHtml } from "hast-util-to-html";
const props = defineProps({
content: {
type: String,
required: true,
default: "# Please input Markdown content",
},
});
// Store the converted HTML string
const renderedHtml = ref("");
// 1. Build pipeline (reuse logic, avoid repeated initialization)
const createProcessor = () => {
return unified()
.use(remarkParse) // MD → MD AST
.use(remarkRehype, { allowDangerousHtml: false }) // MD AST → HTML AST
.use(rehypeHighlight); // Code highlighting (add class to HTML AST)
};
// 2. Core function for MD to HTML conversion (using synchronous method)
const convertMdToHtml = (mdContent) => {
try {
const processor = createProcessor();
// Synchronous processing
const tree = processor.runSync(processor.parse(mdContent));
renderedHtml.value = toHtml(tree, { allowDangerousHtml: false });
} catch (error) {
// Error handling (e.g., MD syntax error)
renderedHtml.value = `<div class="error">Markdown parsing error: ${error.message}</div>`;
}
};
// 3. Watch content changes, update preview in real-time (execute once on initial render)
watch(
() => props.content,
(newContent) => convertMdToHtml(newContent),
{ immediate: true }
);
// 4. Import code highlighting styles (load in onMounted to reduce initial bundle size)
onMounted(() => {
// Can be replaced with other themes (e.g., atom-one-dark.css, monokai.css, github-dark.css)
import("highlight.js/styles/github.css");
});
</script>
3. Using the Component (src/views/HomeView.vue)
<template>
<div class="demo-page">
<h1 class="page-title">Vue + UnifiedJS Real-time Markdown Preview</h1>
<div class="editor-container">
<!-- Left Editor -->
<div class="editor-panel">
<h2 class="panel-title">
<span class="icon">✏️</span>
Markdown Editor
</h2>
<textarea
v-model="markdownContent"
class="markdown-editor"
placeholder="Enter Markdown content here..."
></textarea>
</div>
<div class="divider"></div>
<!-- Right Preview -->
<div class="preview-panel">
<h2 class="panel-title">
<span class="icon">👁️</span>
Real-time Preview
</h2>
<div class="preview-wrapper">
<MarkdownPreview :content="markdownContent" />
</div>
</div>
</div>
<!-- Example Buttons -->
<div class="examples">
<h3>Quick Examples:</h3>
<button
v-for="example in examples"
:key="example.name"
@click="loadExample(example.content)"
class="example-btn"
>
{{ example.name }}
</button>
</div>
</div>
</template>
<script setup>
import { ref } from 'vue'
import MarkdownPreview from '../../components/MarkdownPreview.vue'
// Initial Markdown content
const markdownContent = ref(`# Vue + UnifiedJS in Practice
This is a Markdown preview example **with code highlighting**.
## 1. Features
- ✅ **Real-time Preview**: Render as you type
- ✅ **Code Highlighting**: Supports multiple programming languages
- ✅ **Component-based**: Reusable Vue component
- ✅ **Security**: Prevents XSS attacks
## 2. Code Examples
### JavaScript Example
\`\`\`js
function fibonacci(n) {
if (n <= 1) return n;
return fibonacci(n - 1) + fibonacci(n - 2);
}
console.log(fibonacci(10)); // Output: 55
\`\`\`
### Python Example
\`\`\`python
def quicksort(arr):
if len(arr) <= 1:
return arr
pivot = arr[len(arr) // 2]
left = [x for x in arr if x < pivot]
middle = [x for x in arr if x == pivot]
right = [x for x in arr if x > pivot]
return quicksort(left) + middle + quicksort(right)
print(quicksort([3, 6, 8, 10, 1, 2, 1]))
\`\`\`
## 3. Other Elements
### Blockquote
> This is a blockquote.
> It can be used to display important information or quote others.
### Table
| Language | Type | Feature |
|------|------|------|
| JavaScript | Dynamic | Flexible, widely used |
| Python | Dynamic | Concise, easy to learn |
| TypeScript | Static | Type-safe |
### Links and Images
Visit the [Unified official website](https://unifiedjs.com/) for more information.
### Inline Code
Use \`npm install unified\` to install the core package.
---
**Tip**: Modify the content in the left editor, and the right side will update the preview in real-time!
`)
// Preset examples
const examples = [
{
name: 'Basic Example',
content: `# Heading Example
## Second-level Heading
This is a paragraph of normal text, containing **bold** and *italic*.
- List item 1
- List item 2
- List item 3
> This is a blockquote`
},
{
name: 'Code Example',
content: `# Code Highlighting Example
## JavaScript
\`\`\`js
const greeting = (name) => {
return \`Hello, \${name}!\`;
}
console.log(greeting('World'));
\`\`\`
## Python
\`\`\`python
def greet(name):
return f"Hello, {name}!"
print(greet("World"))
\`\`\``
},
{
name: 'Table Example',
content: `# Table Example
## Fruit Price List
| Fruit | Price (Yuan/Jin) | Origin |
|------|------------|------|
| Apple | 5 | Shandong |
| Banana | 3 | Hainan |
| Orange | 4 | Jiangxi |
## Description
Tables support automatic alignment and formatting.`
}
]
// Load example
const loadExample = (content) => {
markdownContent.value = content
}
</script>
4. Demo Effect
4. UnifiedJS Advanced: Performance Optimization and Ecosystem Expansion
1. Performance Optimization: Handling Large Files and High-Frequency Conversions
1.1 Caching Pipeline Instances (Avoiding Repeated Initialization)
When Markdown needs to be converted frequently (e.g., real-time preview in an editor), repeatedly creating the processor wastes performance. This can be optimized by caching the instance using a "singleton pattern":
// Optimization in MarkdownPreview.vue
let cachedProcessor = null; // Cache processor instance
const getProcessor = () => {
if (!cachedProcessor) {
cachedProcessor = unified()
.use(remarkParse)
.use(remarkRehype)
.use(rehypeHighlight)
.use(rehypeHtml);
}
return cachedProcessor;
};
// Call cached instance in conversion function
const convertMdToHtml = async (mdContent) => {
try {
const processor = getProcessor(); // Reuse cached instance
const result = await processor.process(mdContent);
renderedHtml.value = String(result);
} catch (error) {
renderedHtml.value = `<div class="error">Parsing error: ${error.message}</div>`;
}
};
Optimization Effect: For high-frequency conversions (e.g., triggered 5 times per second), performance improves by about 40% (reducing initialization overhead).
1.2 Chunked Processing for Large Files (Avoiding Blocking)
When processing Markdown over 100,000 words (like long documents), synchronous conversion blocks the main thread, causing page lag. This can be optimized through "chunked processing + Web Worker":
// 1. Create Web Worker (src/workers/mdProcessor.js)
import { unified } from 'unified';
import remarkParse from 'remark-parse';
import remarkRehype from 'remark-rehype';
import rehypeHtml from 'rehype-html';
// Listen for messages from the main thread
self.onmessage = async (e) => {
const { mdContent } = e.data;
try {
// Chunked processing: Split by paragraphs (avoids parsing large text all at once)
const mdChunks = mdContent.split('\n\n'); // Split paragraphs by blank lines
const htmlChunks = [];
const processor = unified()
.use(remarkParse)
.use(remarkRehype)
.use(rehypeHtml);
// Parse paragraph by paragraph, send progress to main thread after each chunk
for (let i = 0; i < mdChunks.length; i++) {
if (mdChunks[i].trim()) {
const result = await processor.process(mdChunks[i]);
htmlChunks.push(String(result));
// Send progress (e.g., "30% complete")
self.postMessage({
type: 'progress',
progress: Math.round((i + 1) / mdChunks.length * 100)
});
}
}
// Processing complete, send final HTML
self.postMessage({
type: 'complete',
html: htmlChunks.join('\n\n')
});
} catch (error) {
self.postMessage({ type: 'error', message: error.message });
}
};
// 2. Use Web Worker in Vue component
// Modify convertMdToHtml function in MarkdownPreview.vue
const convertMdToHtml = async (mdContent) => {
// Only use Web Worker for large files (>10000 characters)
if (mdContent.length > 10000) {
// Create Worker instance
const worker = new Worker(new URL('@/workers/mdProcessor.js', import.meta.url), {
type: 'module'
});
// Listen for Worker messages
worker.onmessage = (e) => {
if (e.data.type === 'progress') {
// Show progress (e.g., progress bar)
console.log(`Parsing progress: ${e.data.progress}%`);
} else if (e.data.type === 'complete') {
renderedHtml.value = e.data.html;
worker.terminate(); // Close Worker
} else if (e.data.type === 'error') {
renderedHtml.value = `<div class="error">Parsing error: ${e.data.message}</div>`;
worker.terminate();
}
};
// Send Markdown content to Worker
worker.postMessage({ mdContent });
} else {
// Still use synchronous processing for small files (more efficient)
const processor = getProcessor();
const result = await processor.process(mdContent);
renderedHtml.value = String(result);
}
};
2. Ecosystem Expansion: Common UnifiedJS Plugins and Scenarios
The core advantage of UnifiedJS is its "plugin ecosystem." Here are common plugin combinations for different scenarios, ready for direct reuse:
2.1 Documentation Generation Scenario (e.g., Component Library Docs)
Requirement: Markdown to HTML + Auto-generate TOC + Code Highlighting + Image Lazy Loading
const processor = unified()
.use(remarkParse) // MD → AST
.use(remark-toc, { heading: 'Table of Contents' }) // Auto-generate TOC (matches "Table of Contents" heading)
.use(remark-add-img-lazy) // Image lazy loading
.use(remarkRehype) // MD AST → HTML AST
.use(rehype-highlight) // Code highlighting
.use(rehype-slug) // Add IDs to headings (for TOC anchor jumps)
.use(rehype-autolink-headings) // Add anchor links to headings (click to jump to itself)
.use(rehype-html); // HTML AST → HTML string
2.2 WeChat Official Account Formatting Scenario
Requirement: Markdown to HTML conforming to WeChat Official Account style (adapted for mobile)
const processor = unified()
.use(remarkParse)
.use(remarkRehype)
.use(rehype-highlight)
.use(rehype-weixin-style) // WeChat style adaptation (e.g., font, line height)
.use(rehype-img-process, { // Image processing: convert to WeChat temporary link/add watermark
watermark: 'My Official Account', // Add watermark text
maxWidth: 677 // Maximum width for WeChat images
})
.use(rehype-html);
5. Summary
UnifiedJS is not "another Markdown tool," but a "standardized framework for frontend content processing" — through its "AST-driven + plugin-based" design, it solves the pain points of traditional tools like "format fragmentation and difficult logic reuse," allowing developers to "process all text formats with a single workflow."
With the development of frontend documentation tools, low-code platforms, and content management systems, the UnifiedJS ecosystem will continue to improve: on one hand, the official team will optimize the "newcomer experience" (e.g., simplifying configuration, adding scenario-based templates); on the other hand, the community will produce more plugins for vertical scenarios (e.g., AI content processing, multi-language translation).
For developers, mastering UnifiedJS not only improves the efficiency of "content processing" but also fosters a mindset of "structurally processing text" — a mindset equally applicable in scenarios like editor development and data cleaning, making it an important skill for frontend engineers.
If you encounter problems in practice, feel free to discuss them in the comments section, or get help through the UnifiedJS GitHub community (UnifiedJS GitHub).
Top 1 of 2 from juejin.cn, machine-translated. The original thread is authoritative.
Great!!!
[Rose]