A Pure Frontend Toolkit Downloads Offline Map Tiles and Clips Them by Administrative Boundary
theme: fancy highlight: vs
Previously, I wrote an article "No Cost! Pure Frontend Package and Download Offline Tile Maps" about crawling open tile basemaps, writing them into a jszip archive, and downloading offline tile basemaps.
Now it's time for an upgrade, summarizing the download of some commonly used map basemaps.
1. Coordinate Systems
Currently, the commonly used coordinate systems in China are:
- National Geodetic Coordinate System CGCS2000 (EPSG:4547)
- Baidu Coordinate System BD09
- Mars Coordinate System GCJ02
- International Universal Coordinate System WGS84 (EPSG:4326)
For more geographic coordinate systems, you can search at https://epsg.io/ to get related configurations.
EPSG geographic coordinate systems can generally be transformed using proj4, for example:
Mutual transformation between the National Geodetic Coordinate System CGCS2000 and the International Universal Coordinate System WGS84
import proj4 from "proj4";
// CGCS2000 Geographic Coordinate System
const cgcs2000 = "+proj=tmerc +lat_0=0 +lon_0=114 +k=1 +x_0=500000 +y_0=0 +ellps=GRS80 +units=m +no_defs";
// WGS84 Geographic Coordinate System
const wgs84 = "+proj=longlat +datum=WGS84 +no_defs";
{
// Transform CGCS2000 coordinates to WGS84 coordinates
const [lng, lat] = proj4(cgcs2000, wgs84, [495741.2653999999, 2499953.2905]);
console.log(lng, lat); //113.95858231900118 22.597395765246997
}
{
// Transform WGS84 coordinates to CGCS2000 coordinates
const [lng, lat] = proj4(wgs84, cgcs2000, [113, 22]);
console.log(lng, lat); //396734.05319196207 2434138.0838125898
}
Baidu Coordinate System BD09 and Mars Coordinate System GCJ02 can be transformed using gcoord
import gcoord from "gcoord";
{
// Transform WGS84 coordinates to BD09
const [lng, lat] = gcoord.transform([113, 22], gcoord.WGS84, gcoord.BD09);
console.log(lng, lat); //113.01173112892457 22.00318110953146
}
{
// Transform WGS84 coordinates to GCJ02
const [lng, lat] = gcoord.transform([113, 22], gcoord.WGS84, gcoord.GCJ02);
console.log(lng, lat); //113.00519647627083 21.997261931400583
}
Utility Class Create map projection
const {name, config, origin, resolutions} = projConfig;
// Define projection
proj4.defs(name, config);
const scales: number[] = [];
// lods configuration
if (resolutions?.length) {
for (let i = resolutions.length - 1; i >= 0; i--) {
if (resolutions[i]) {
scales[i] = 1 / resolutions[i];
}
}
}
Coordinate transformation
// Coordinate offset
transformation: (function () {
// Origin position offset
if (origin?.length) {
return new Transformation(1, -origin[0], -1, origin[1]);
}
const scale = 0.5 / (Math.PI * EARTH_R);
return new Transformation(scale, 0.5, -scale, 0.5);
})(),
// Project coordinates
project(lnglat: LngLatXY): LngLatXY {
return proj4(name).forward(lnglat);
},
// Unproject coordinates
unproject(xy: LngLatXY): LngLatXY {
return proj4(name).inverse(xy);
},
// Convert longitude/latitude to pixel coordinates
lnglat2px(lnglat: LngLatXY, zoom: number): LngLatXY {
const p = this.project(lnglat);
const scale = this.scale(zoom);
return this.transformation.transform(p, scale);
},
// Convert pixel coordinates to longitude/latitude
px2lnglat(xy: LngLatXY, zoom: number): LngLatXY {
const scale = this.scale(zoom);
const p = this.transformation.untransform(xy, scale);
return this.unproject(p);
},
Zoom level and pixel size can refer to the code of leaflet and proj4leaflet, defaulting to spherical Mercator projection.
Calculate the pixel size for the zoom level
scale(zoom: number) {
// Pixel size for lods levels
if (resolutions?.length) {
let iZoom = Math.floor(zoom),
baseScale,
nextScale,
scaleDiff,
zDiff;
if (zoom === iZoom) {
return scales[zoom];
} else {
// Non-integer zoom, interpolate
baseScale = scales[iZoom];
nextScale = scales[iZoom + 1];
scaleDiff = nextScale - baseScale;
zDiff = zoom - iZoom;
return baseScale + scaleDiff * zDiff;
}
}
return tileSize * Math.pow(2, zoom);
}
Calculate the zoom level corresponding to the pixel size
zoom(scale: number) {
// lods zoom level
if (resolutions?.length) {
// Find closest number in this._scales, down
let downScale = closestElement(scales, scale);
let downZoom = scales.indexOf(downScale!),
nextScale,
nextZoom,
scaleDiff;
// Check if scale is downScale => return array index
if (scale === downScale) {
return downZoom;
}
if (downScale === undefined) {
return -Infinity;
}
// Interpolate
nextZoom = downZoom + 1;
nextScale = scales[nextZoom];
if (nextScale === undefined) {
return Infinity;
}
scaleDiff = nextScale - downScale;
return (scale - downScale) / scaleDiff + downZoom;
}
return Math.log(scale / tileSize) / Math.LN2;
},
2. Tile Basemap Download
2.1 Handwritten Canvas Map
Get the Image object for a tile
getTileImage(x: number, y: number, z: number) {
return new Promise<HTMLImageElement | null>((resolve, reject) => {
const id = `${x}-${y}-${z}`;
// Cache tile basemap
if (this.cacheTiles[id] !== undefined) {
resolve(this.cacheTiles[id]);
} else {
// Load tile basemap
const url = this.tileUrl
.replace('{x}', String(x))
.replace('{y}', String(y))
.replace('{z}', String(z));
const image = new Image();
image.src = url;
image.crossOrigin = 'anonymous';
image.onload = () => {
this.cacheTiles[id] = image;
resolve(image);
};
image.onerror = () => {
this.cacheTiles[id] = null;
reject(image);
};
}
});
}
Draw tile Image on canvas
async drawTileImage(
ctx: CanvasRenderingContext2D,
x: number,
y: number,
z: number,
imageX: number,
imageY: number
) {
try {
const image = await this.getTileImage(x, y, z);
if (image) {
ctx.drawImage(image, imageX, imageY);
}
} catch (error) {}
}
Calculate tile bounds and related parameters
getTileBounds(center?: LngLatXY, zoom?: number) {
// Convert center longitude/latitude to pixel coordinates
const tileCenter = this.lnglat2xy(center ?? this.center, zoom ?? this.zoom);
// Canvas size
const mapSize = this.getMapSize();
// Take half to get the pixel coordinates of the top-left and bottom-right points relative to the center point
const halfWidth = mapSize[0] * 0.5;
const halfHeight = mapSize[1] * 0.5;
const start: LngLatXY = [tileCenter[0] - halfWidth, tileCenter[1] - halfHeight];
const end: LngLatXY = [tileCenter[0] + halfWidth, tileCenter[1] + halfHeight];
// Tile basemap is an image of tileSize x tileSize, calculate tile bounds
const bounds = [
[Math.floor(start[0] / this.tileSize), Math.floor(start[1] / this.tileSize)],
[Math.ceil(end[0] / this.tileSize), Math.ceil(end[1] / this.tileSize)]
];
return {
tileCenter,
bounds,
start,
end,
// Offset of the tile start pixel coordinate relative to the top-left pixel coordinate of the canvas visible range
offset: [bounds[0][0] * this.tileSize - start[0], bounds[0][1] * this.tileSize - start[1]]
};
}
Draw tile basemap
- Calculate the map's visible range, collect tile indices and positions
- Sort the drawing order of tiles, prioritizing drawing cached tiles
- Trigger 6 concurrent requests to load images and draw them onto the canvas
async drawLayer() {
const ctx = this.ctx;
ctx.clearRect(0, 0, this.canvas.width, this.canvas.height);
const {offset, bounds, start, end, tileCenter} = this.getTileBounds();
this.tileCenter = tileCenter;
// Start pixel coordinates
this.tileStart = start;
// End pixel coordinates
this.tileEnd = end;
// Collect tile indices to be drawn and their positions on the canvas
const queue = [];
for (let x = bounds[0][0], i = 0; x < bounds[1][0]; x++, i++) {
for (let y = bounds[0][1], j = 0; y < bounds[1][1]; y++, j++) {
queue.push({
x,
y,
imageX: i * this.tileSize + offset[0],
imageY: j * this.tileSize + offset[1]
});
}
}
// Sort to prioritize drawing cached tiles
queue.sort((a: any, b: any) => {
const id1 = `${a.x}-${a.y}-${this.zoom}`;
const id2 = `${b.x}-${b.y}-${this.zoom}`;
if (this.cacheTiles[id1]) return -1;
if (this.cacheTiles[id2]) return 1;
return 0;
});
// Asynchronously load images and draw them onto the canvas. Under HTTP/1.1, TCP concurrent connections for the same domain are 4-8, typically 6.
for (let i = 0; i < queue.length; i = i + 6) {
const list = queue.slice(i, i + 6);
await Promise.all(
list.map((a) => this.drawTileImage(ctx, a.x, a.y, this.zoom, a.imageX, a.imageY))
);
}
// Draw shapes
this.drawShape();
}
The following map tiles use Tianditu
Additionally, you can set the map projection you need. If using an ArcGIS map service, you can get tileInfo via /MapServer?f=json and configure the origin, zoom levels (lods), tile size, max/min zoom levels, projection coordinate system, etc. Projection coordinate system configurations can be searched at https://epsg.io/.
Below is the ArcGIS map service configuration for the National Geodetic Coordinate System 2000 in one of my projects
"tileInfo": {
"rows": 1024, // Tile size
"cols": 1024, // Tile size
"dpi": 96,
"format": "PNG32",
"compressionQuality": 0,
"origin": { // Origin
"x": -5123200,
"y": 10002100
},
"spatialReference": { // Projection Coordinate System
"wkid": 4547,
"latestWkid": 4547
},
"lods": [ // Zoom Levels
{
"level": 0,
"resolution": 132.291931250529,
"scale": 500000
},
{
"level": 1,
"resolution": 79.3751587503175,
"scale": 300000
},
{
"level": 2,
"resolution": 66.1459656252646,
"scale": 250000
},
//...
]
},
2.2 Download Tiles
After selecting a range, get the tile URLs and xyz for the levels to be downloaded within that range
getTileList(rect: [LngLatXY, LngLatXY], zoom: number) {
const p1: LngLatXY = this.lnglat2xy(rect[0], zoom);
const p2: LngLatXY = this.lnglat2xy(rect[1], zoom);
const start = [Math.min(p1[0], p2[0]), Math.min(p1[1], p2[1])];
const end = [Math.max(p1[0], p2[0]), Math.max(p1[1], p2[1])];
// Calculate tile bounds
const bounds = [
[Math.floor(start[0] / this.tileSize), Math.floor(start[1] / this.tileSize)],
[Math.ceil(end[0] / this.tileSize), Math.ceil(end[1] / this.tileSize)]
];
const queue: any[] = [];
for (let x = bounds[0][0], i = 0; x < bounds[1][0]; x++, i++) {
for (let y = bounds[0][1], j = 0; y < bounds[1][1]; y++, j++) {
const url = this.tileUrl
.replace('{x}', String(x))
.replace('{y}', String(y))
.replace('{z}', String(zoom));
queue.push({
x,
y,
z: zoom,
url
});
}
}
return queue;
}
If you need to download large zoom levels, the number of tiles may be too large, causing overflow. In this case, split-package downloading is needed.
To use split-package downloading, you need to disable the browser's "Ask where to save each file before downloading" and grant the webpage permission for "Automatic downloads"
download: async () => {
const {minLevel, maxLevel} = state.value;
if (minLevel > maxLevel) {
ElMessage.error('The minimum level for download must be less than or equal to the maximum level!');
return;
}
const b = state.value.bounds;
if (checkBounds(b as [LngLatXY, LngLatXY])) {
const queue: any[] = [];
try {
const rect = b as [LngLatXY, LngLatXY];
for (let z = minLevel; z <= maxLevel; z++) {
const q = map.getTileList(rect, z);
queue.push(...q);
}
} catch (error) {
ElMessage.error('Too many tiles, please download by level');
return;
}
if (!window.confirm(`Need to download ${new Intl.NumberFormat().format(queue.length)} tile basemaps, estimated download time ${getTime(queue.length * 0.5)} seconds`))
return;
store.value.current = 0;
store.value.total = queue.length;
store.value.loading = true;
// Split-package download
if (state.value.isSplit) {
const n = state.value.spliteNum;
for (let i = 0; i < queue.length; i = i + n) {
const list = queue.slice(i, i + n);
console.log(i, i + n, list.length);
await downloadZip(list, i);
}
} else {
await downloadZip(queue, 0);
}
store.value.loading = false;
} else {
ElMessage.error('Please select a range');
}
},
Get image data and write it into a zip package.
export const downloadZip = (queue: any[], start: number) => {
return new Promise(async (resolve) => {
const {minLevel, maxLevel} = state.value;
const zip = new JSZip();
// Asynchronously load images and draw them onto the canvas. Under HTTP/1.1, TCP concurrent connections for the same domain are 4-8, typically 6.
for (let i = 0; i < queue.length; i += 6) {
const list = queue.slice(i, i + 6);
store.value.current = start + i;
await Promise.all(list.map((a) => writeZip(zip, a.url, a.x, a.y, a.z)));
await sleep();
}
zip
.generateAsync({type: 'blob'})
.then(function (content) {
downloadFile(
content,
`Tile Levels [${minLevel}-${maxLevel}][${start}]${new Date().getTime()}.zip`
);
})
.finally(() => {
resolve(start);
});
});
};
For downloaded split-package tiles, just "Extract to current location". The tiles folder will contain the complete collection of all split-package downloaded tile images.
3. Regional Boundary Map Download
Get administrative region boundaries, draw them, and get the administrative region range
drawarea: (noFit: boolean) => {
const code = state.value.currentArea;
if (!code) {
ElMessage.error('Please select a region');
return;
}
const drawArea = (res: any) => {
if (areaIds.length) {
areaIds.forEach((id) => {
map.removeShape(id);
});
areaIds = [];
}
let count = 0,id = 0;
let lng = 0,lat = 0;
const bounds = [[Number.MAX_SAFE_INTEGER, Number.MAX_SAFE_INTEGER], [0, 0]] as [LngLatXY, LngLatXY];
theGeojson = res;
map.isGc = state.value.isGc;
travelGeo(res, (path: any) => {
id++;
const ps: any[] = [];
path.forEach((a: any) => {
const b = state.value.isGc ? a : gcTowgs84(a[0], a[1]);
ps.push(b);
count++;
lng += b[0];
lat += b[1];
bounds[0][0] = Math.min(bounds[0][0], b[0]);
bounds[0][1] = Math.min(bounds[0][1], b[1]);
bounds[1][0] = Math.max(bounds[1][0], b[0]);
bounds[1][1] = Math.max(bounds[1][1], b[1]);
});
const s = code + '_' + id;
areaIds.push(s);
map.addShape({
id: s,
type: 'polygon',
path: ps,
style: {
stroke: true,
color: 'blue',
opacity: 1,
weight: 2,
fill: true,
fillColor: 'blue',
fillOpacity: 0.1
}
});
});
state.value.areaBounds = bounds;
state.value.areaCenter = [lng / count, lat / count];
if (!noFit) map.fitBounds({bounds, paddingLeft: 300});
state.value.center = map.getCenter();
state.value.zoom = map.getZoom();
state.value.currentArea = code;
};
const id = `${code}${state.value.isFull ? '_full' : ''}`;
if (cacheGeo[id]) {
drawArea(cacheGeo[code]);
return;
}
fetch(`https://geo.datav.aliyun.com/areas_v3/bound/${id}.json`)
.then((res) => res.json())
.then((res) => {
cacheGeo[id] = res;
drawArea(res);
}).catch((err) => {
ElMessage.error('Failed to get regional boundary, please reselect');
});
}
Map tiles are sourced from Amap, and regional boundary data comes from the GeoJSON of the DataV.GeoAtlas Geographic Tool Series, which is based on the Mars coordinate system.
Capture the canvas map content within the administrative region and download the map for that administrative region.
async drawAreaCanvas(geojson: any, rect: [LngLatXY, LngLatXY], zoom: number) {
const canvas = document.createElement('canvas');
const ctx = canvas.getContext('2d')!;
const tileSize = this.tileSize;
const {bounds, offset, start, end} = this.getTileInfo(rect, zoom);
canvas.width = end[0] - start[0];
canvas.height = end[1] - start[1];
// Draw mask
const maskPath = new Path2D();
travelGeo(geojson, (paths: Array<[number, number]>) => {
const r = new Path2D();
paths.forEach((a, index: number) => {
const b = this.isGc ? a : gcTowgs84(a[0], a[1]);
const p = this.lnglat2xy(b, zoom);
const point = [p[0] - start[0], p[1] - start[1]];
if (index === 0) r.moveTo(point[0], point[1]);
else r.lineTo(point[0], point[1]);
});
r.closePath();
maskPath.addPath(r);
});
const queue = [];
for (let x = bounds[0][0], i = 0; x < bounds[1][0]; x++, i++) {
for (let y = bounds[0][1], j = 0; y < bounds[1][1]; y++, j++) {
queue.push({
x,
y,
imageX: i * tileSize + offset[0],
imageY: j * tileSize + offset[1]
});
}
}
// Sort to prioritize drawing cached tiles
queue.sort((a: any, b: any) => {
const id1 = `${a.x}-${a.y}-${zoom}`;
const id2 = `${b.x}-${b.y}-${zoom}`;
if (this.cacheTiles[id1]) return -1;
if (this.cacheTiles[id2]) return 1;
return 0;
});
// Asynchronously load images and draw them onto the canvas. Under HTTP/1.1, TCP concurrent connections for the same domain are 4-8, typically 6.
for (let i = 0; i < queue.length; i = i + 6) {
const list = queue.slice(i, i + 6);
await Promise.all(
list.map((a) => this.drawTileImage(ctx, a.x, a.y, zoom, a.imageX, a.imageY))
);
}
// Capture within administrative region
ctx.globalCompositeOperation = 'destination-in';
ctx.fillStyle = '#000';
ctx.fill(maskPath);
return {canvas, queue};
}
Note: Due to canvas size limitations in the browser, it is impossible to capture administrative region maps at larger zoom levels.
4. Download Administrative Region Tile Maps
- Modify based on the code above
- Change the canvas size to
index range * tileSize
canvas.width = (bounds[1][0] - bounds[0][0]) * tileSize;
canvas.height = (bounds[1][1] - bounds[0][1]) * tileSize;
- Modify the drawing coordinates
// Capture the mask coordinates of the administrative region, reverse offset
const point =[p[0] - start[0] - offset[0], p[1] - start[1] - offset[1]]
// Tile index and coordinates on canvas, no longer need offset
queue.push({
x,
y,
imageX: i * tileSize
imageY:j * tileSize
});
- Disable cached tile sorting to avoid index order confusion
- Sequentially cut the canvas into images of
tileSizewidth and height based on index coordinates, and write them into a zip package
export const splitMapCanvas = (
zip: JSZip,
canvas: HTMLCanvasElement,
zoom: number,
tileSize: number,
queue: Array<{x: number; y: number}>
) => {
let idx = 0;
for (let x = 0; x < canvas.width; x += tileSize) {
for (let y = 0; y < canvas.height; y += tileSize) {
const {x: x1, y: y1} = queue[idx];
const tempCanvas = document.createElement('canvas');
tempCanvas.width = tileSize;
tempCanvas.height = tileSize;
const tempctx = tempCanvas.getContext('2d')!;
tempctx.drawImage(canvas, x, y, tileSize, tileSize, 0, 0, tileSize, tileSize);
const base64 = tempCanvas.toDataURL('image/png');
const file = convertBase64UrlToFile(base64, zoom + '.png');
zip.file(`tiles/${zoom}/${y1}/${x1}.png`, file);
idx++;
}
}
};
5. Verification
A grid index can be drawn to facilitate verification of whether the tile positions in the downloaded zip package are correct.
You can also directly modify the tile URL path for verification.
Of course, you can also use Leaflet or OpenLayers for verification.
//leaflet
const map = new L.map('container', {
attributionControl: false,
doubleClickZoom: false,
preferCanvas: true,
})
map.setView([22.629045999999985, 114.0869626738281], 11)
L.tileLayer(
'/demo/tiles/{z}/{y}/{x}.png',
{
tileSize: 256,
}
).addTo(map)
//OpenLayers
new ol.Map({
target: 'map',
layers: [
new ol.layer.Tile({
source: new ol.source.XYZ({
url: '/demo/tiles/{z}/{y}/{x}.png',
tileSize: 256
}),
})
],
view: new ol.View({
center: ol.proj.fromLonLat([114.0869626738281, 22.629045999999985]),
zoom: 11
})
});
6. GitHub Address and Access Address
https://github.com/xiaolidan00/offline-map-download
References
- jszip- https://stuk.github.io/jszip/
- leaflet- https://github.com/Leaflet/Leaflet
- proj4- https://proj4js.org/
- proj4leaflet- https://kartena.github.io/Proj4Leaflet/
- Tianditu- http://lbs.tianditu.gov.cn/server/MapService.html
- Amap- https://lbs.amap.com/api/javascript-api-v2/summary
- DataV.GeoAtlas Geographic Tool Series- https://datav.aliyun.com/portal/school/atlas/area_selector
- Projection Coordinate Systems- https://epsg.io/
- OpenLayers- https://openlayers.org/