Google's 5.5MB Pose Model Runs at 60fps in the Browser, No Backend

My plan is to recognize singing 🎤, dancing 🕺, rap 🗣️, and basketball 🏀. Today I'm recognizing dancing 🕺

The video material is AI-generated, used only for testing.

Even with a pure CPU, the video doesn't lag. I'm using an AMD 5500, an entry-level CPU, but video recognition still hits around 60 frames per second.

Upload an image to draw a skeleton, play a video and the skeleton follows along, or open the camera for real-time recognition. Runs purely in the browser, zero backend. The model is only 5.5MB, and a single model handles images, video, and camera input.

Flowchart

    Open page
         │
         ▼
    Load WASM runtime
         │
         ▼
    Load 5.5MB model in background
         │
         ▼
    Display "Ready · GPU"
         │
         ▼
    Select Image / Video / Camera
         │
         ▼
    detectForVideo inference
         │
         ▼
    33 keypoints + confidence
         │
         ▼
    Canvas draws skeleton

10 lines to run:

import * as $mp from "https://cdn.jsdelivr.net/npm/@mediapipe/[email protected]/+esm"
const { FilesetResolver, PoseLandmarker } = $mp
const vision = await FilesetResolver.forVisionTasks(
  "https://cdn.jsdelivr.net/npm/@mediapipe/[email protected]/wasm"
)
const pose = await PoseLandmarker.createFromOptions(vision, {
  baseOptions: { modelAssetPath: "./models/pose_landmarker_lite.task", delegate: "GPU" },
  runningMode: "VIDEO",
})
const r = pose.detectForVideo(img, performance.now())
console.log(r.landmarks[0].length) // 33

@mediapipe/tasks-vision, made by Google, has 17k+ stars on GitHub. Traditional solutions require installing Python or setting up a GPU server. MediaPipe moves all of this into the browser.

1. Model Preloading + GPU/CPU Switching

async function loadModel(delegate) {
  const vision = await FilesetResolver.forVisionTasks(
    "https://cdn.jsdelivr.net/npm/@mediapipe/[email protected]/wasm"
  )
  poseLandmarker = await PoseLandmarker.createFromOptions(vision, {
    baseOptions: { modelAssetPath: "./models/pose_landmarker_lite.task", delegate },
    runningMode: "VIDEO", numPoses: 1,
  })
}
loadModel("GPU") // Load as soon as the page opens

async function switchDelegate(delegate) {
  currentDelegate = delegate
  await loadModel(delegate) // Rebuild model after switching
}

Pass "GPU" to delegate to use WebGL, or "CPU" to run natively. With a small model, CPU can sometimes be faster; just click a button to switch.

Block actions until the model is loaded:

if (!poseLandmarker) { setStatus("Model not loaded yet, please wait"); return }

2. One Model for Images + Video

VIDEO mode handles both input types; treat an image as a single frame with a timestamp:

// Image
const r = poseLandmarker.detectForVideo(img, performance.now())
// Video frame
const r2 = poseLandmarker.detectForVideo(video, performance.now())

A single detection takes about 25ms:

Image: 800 × 600 | People: 1 | Inference: 35ms

3. Video/Camera, Throttled to 20fps

Don't run inference on every frame. Use requestAnimationFrame with a 50ms throttle:

function loopFrames(videoEl, cctx) {
  const now = performance.now()
  if (now - lastDetect > 50) {
    const results = poseLandmarker.detectForVideo(videoEl, now)
    cctx.drawImage(videoEl, 0, 0, W, H)
    drawSkeleton(cctx, W, H, results.landmarks[0])
    lastDetect = now
  }
  raf = requestAnimationFrame(() => loopFrames(videoEl, cctx))
}

The same loop works for the camera:

stream = await navigator.mediaDevices.getUserMedia({ video: { width: 640, height: 480 } })
camVideo.srcObject = stream
loopFrames(camVideo, camctx)

Note: localhost or https is required to access the camera. Add playsinline to the video element for iOS compatibility.

4. Drawing the Skeleton + Standing/Sitting/Lying Detection

Four color-coded connection groups:

const CONNECTIONS = {
  spine: [[11,12],[11,23],[12,24],[23,24]],
  left:  [[11,13],[13,15],[15,17],[17,19],[19,21],[21,22],[21,25],[25,27],[27,29],[29,31],[31,32]],
  right: [[12,14],[14,16],[16,18],[18,20],[20,22],[22,26],[26,28],[28,30],[30,32]],
  head:  [[0,1],[1,2],[2,3],[3,7],[7,0],[0,4],[4,5],[5,6],[6,8],[8,0]]
}

for (const [i, j] of CONNECTIONS.spine) {
  ctx.moveTo(person[i].x * W, person[i].y * H)
  ctx.lineTo(person[j].x * W, person[j].y * H)
}
ctx.lineWidth = Math.max(2, W * 0.005) // Adaptive line width
ctx.strokeStyle = "#ffb84d"
ctx.stroke()

Standing/sitting/lying detection uses the y-coordinate difference between shoulders and hips:

const hipY = (person[23].y + person[24].y) / 2
const shoulderY = (person[11].y + person[12].y) / 2
const gap = hipY - shoulderY

if (gap > 0.12) return "Standing"
if (gap > 0.04) return "Sitting"
return "Lying down"

Note that y increases downward. Writing shoulderY - hipY would produce a negative number and always return "Lying down".

6. Outputting 33 Keypoint Coordinates

[
  { x: 0.52, y: 0.18, z: 0.01, score: 0.98 }, // 0 nose
  { x: 0.48, y: 0.22, z: 0.01, score: 0.95 }, // 1 left_eye_inner
  ...
]

x and y are normalized from 0 to 1. z is depth. Points with a score below 0.5 are grayed out in the list.

6. Serving the App

WASM requires COOP/COEP headers:

import { join } from "node:path"

Bun.serve({ port: 3005, async fetch(req) {
  const file = Bun.file(join(import.meta.dir, "static",
    new URL(req.url).pathname === "/" ? "/index.html" : new URL(req.url).pathname))
  return new Response(file, { headers: {
    "Cross-Origin-Opener-Policy": "same-origin",
    "Cross-Origin-Embedder-Policy": "require-corp",
  }})
}})

Summary

• • MediaPipe Tasks Vision: Pure JS, 17k+ stars

• • Lite model: 5.5MB, 95% accuracy

• • Page preloading: Loads on open, with a status badge

• • GPU/CPU switching: Test and use whichever is faster

• • VIDEO mode: One model handles both images and video

• • 20fps throttling: Smooth video recognition without lag

• • Four-color skeleton: Head/arms/spine/legs distinguishable at a glance

Previous Articles

Web-based, 6.5MB Face Recognition Model, Google Framework, Fast and Accurate

Web-based OCR, Loads a 6MB Model, Fast and Accurate, Baidu Really Delivers This Time