跪拜 Guibai
← Back to the summary

The Shell Commands You Keep Looking Up: Archiving, Permissions, and the Tools That Glue Everything Together

The first two articles covered files and text, processes and networking — covering more than half of the high-frequency commands encountered daily in shell programming. This article is the final part of the series, covering the remaining three areas: archiving and compression, permissions and users, and some advanced tools.

The characteristic of these three groups of commands is "used infrequently but checked every time." Archiving might be used only a few times a year, permission scripts are used daily but parameters are easily forgotten, and utility commands like xargs and cron can save a lot of trouble when used well. After reading this article, the three-part common commands series is complete.

1. Archiving and Compression

Archiving and compression are two separate actions in the shell: archiving combines multiple files into one (using tar), and compression makes a single file smaller (using gzip, xz, etc.). In daily use, we use tar to do both simultaneously — tar itself does not compress, but it can call gzip or xz to do it together.

1.1 tar: Archiving

tar is an abbreviation for tape archive, originating from the era of tape backups. The core purpose of modern tar is to package a directory into a single file.

# Package
tar -cf archive.tar dir/

# Unpack
tar -xf archive.tar

# View contents
tar -tf archive.tar

These are the most basic usages. But in daily use, we more commonly use "package + compress":

# Package and compress with gzip
tar -czf archive.tar.gz dir/

# Package and compress with xz (smaller but slower)
tar -cJf archive.tar.xz dir/

# Package and compress with bzip2
tar -cjf archive.tar.bz2 dir/

# Decompress (tar automatically detects the compression format)
tar -xzf archive.tar.gz
tar -xJf archive.tar.xz
tar -xjf archive.tar.bz2

Several common options:

Several practical combinations:

# Package and exclude certain files
tar -czf backup.tar.gz --exclude='*.log' --exclude='tmp/' /var/data/

# Package and show progress
tar -czf - dir/ | pv > archive.tar.gz

# Extract to a specified directory
tar -xzf archive.tar.gz -C /opt/

# Extract a single file (without extracting the entire archive)
tar -xzf archive.tar.gz path/inside/file.txt

# Append files to an existing tar (uncommon)
tar -rf archive.tar newfile.txt

A detail about tar: -f must be immediately followed by the filename; you cannot write it separately as -f file.tar. -zcvf and -zcfv are equivalent, but -zvfc is not valid.

1.2 Trade-offs between various compression formats

# gzip: Default choice, balances speed and compression ratio
gzip file.txt          # Compress (deletes the original file)
gunzip file.txt.gz     # Decompress

# bzip2: Higher compression ratio than gzip, slower
bzip2 file.txt
bunzip2 file.txt.bz2

# xz: Highest compression ratio, slowest
xz file.txt
unxz file.txt.xz

# zip: Cross-platform (Windows also recognizes it), average compression ratio
zip -r archive.zip dir/
unzip archive.zip

Trade-offs for compression formats:

Recommendation: Use gzip for daily backups and transfers, xz for release packages and long-term archiving, and zip when sharing with Windows.

1.3 7z: High Compression Ratio

7z is the command-line version of 7-Zip, with a compression ratio slightly higher than xz.

# Install
apt install p7zip-full    # Debian/Ubuntu
yum install p7zip         # CentOS/RHEL

# Compress
7z a archive.7z dir/

# Decompress
7z x archive.7z

7z is suitable for "must be minimal" scenarios (embedded systems, limited network transmission), but is rarely used daily.

2. Permissions and Users

Permissions are the infrastructure of the Linux system. The most frequently encountered permission-related commands in shell scripts are chmod, chown, umask, and sudo. This section explains these clearly.

2.1 Basics of File Permissions

Before discussing the commands, let's quickly review Linux file permissions. The first column of ls -l is the permission bits:

-rw-r--r-- 1 user group 1024 Jan 1 12:00 file.txt
drwxr-xr-x 2 user group 4096 Jan 1 12:00 dir/

Permission bits are interpreted in four groups:

Each group of three bits represents: read r (4), write w (2), execute x (1). The numeric representation is the sum of the three permission bit values.

2.2 chmod: Change Permissions

# Numeric mode
chmod 755 file.sh        # rwxr-xr-x
chmod 644 file.txt       # rw-r--r--
chmod 600 secret.key     # rw-------
chmod 700 private/       # rwx------

# Symbolic mode
chmod u+x file.sh        # Owner adds execute permission
chmod g-w file.txt       # Group removes write permission
chmod o=r file.txt       # Others set to read-only permission
chmod a+r file.txt       # Everyone adds read permission
chmod +x script.sh       # Shorthand: everyone adds execute permission

Several common permissions:

Batch change permissions:

# Add execute permission to all .sh files in a directory
find . -name "*.sh" -exec chmod +x {} \;

# Recursively set directories to 755 and files to 644 (a very common deployment scenario)
find /opt/myapp -type d -exec chmod 755 {} \;
find /opt/myapp -type f -exec chmod 644 {} \;

2.3 chown / chgrp: Change Owner and Group

# Change owner
chown alice file.txt

# Change owner and group
chown alice:developers file.txt

# Change group (only change group)
chgrp developers file.txt

# Recursively change
chown -R alice:developers /opt/myapp/

chown can only be executed by root; a regular user cannot change the owner of their own files. For groups, it depends on whether the user is a member of the group.

2.4 umask: Default Permissions

umask is the "user file creation mask" — it determines the default permissions for newly created files and directories.

# View current umask
umask
# Output: 0022

# Set umask
umask 022

umask algorithm: Default permissions minus umask.

For example, with umask 022:

Recommendation: On servers, umask is usually 0022; for security scenarios, use 0077 (only the owner can view). If you need to strictly control file permissions in a script, you can temporarily change umask:

old_umask=$(umask)
umask 077
# Files created next will have 600 permissions
touch secret.key
umask "$old_umask"

2.5 sudo / su: Privilege Escalation

sudo allows a regular user to execute commands as root (or another user).

# Execute as root
sudo apt update

# Execute as another user
sudo -u alice whoami

# Switch to root shell
sudo -i
sudo su -

# Edit a file (when permissions are needed)
sudo vim /etc/nginx/nginx.conf

# List commands the current user can sudo
sudo -l

Security considerations when using sudo:

su is short for switch user. The difference from sudo is:

Daily use recommends sudo — it is auditable, configurable, and does not require sharing the root password.

2.6 id / groups: Check Identity

# Current user information
id
# Output: uid=1000(alice) gid=1000(alice) groups=1000(alice),27(sudo)

# Only show UID
id -u

# Only show group
id -g

# List all groups
groups

id is very commonly used in scripts — to check if the current user is root:

if [ "$(id -u)" -ne 0 ]; then
    echo "Please run this script as root" >&2
    exit 1
fi

3. Advanced Tools

This last group consists of utility commands. They don't directly perform tasks themselves, but they amplify the capabilities of other commands.

3.1 xargs: Turn stdin into Arguments

xargs is one of the most underestimated commands in the shell. It solves two problems: "command arguments are too long" and "combining commands."

# Most common usage: find + xargs + command
find . -name "*.log" | xargs rm

# Equivalent but safer (handles filenames with spaces)
find . -name "*.log" -print0 | xargs -0 rm

-print0 and -0 work together using the null character as a delimiter — this is the safe mode for handling "filenames with spaces."

# Execute a command once per file
echo "file1 file2 file3" | xargs -n 1 cp -- /tmp/

# Parallel execution
find . -name "*.jpg" | xargs -P 4 -I {} convert {} -resize 50% {}.small.jpg

-P N means "execute in parallel using N processes" — this is the key to using xargs for concurrent tasks.

# Use a placeholder to specify argument position
find . -name "*.sh" | xargs -I {} chmod +x {}

# Handle input containing special characters
echo "name with spaces" | xargs -d '\n' touch

Trade-offs between xargs and find -exec:

Default to the find ... -print0 | xargs -0 ... pattern; it's both fast and safe.

3.2 tee: Kill Two Birds with One Stone

tee outputs stdin to both stdout and a file simultaneously, allowing you to see it on the screen and save it to a log.

# Output to both screen and file
ls -la | tee output.log

# Append (do not overwrite)
ls -la | tee -a output.log

# Use sudo to save to a file requiring permissions
echo "127.0.0.1 example.com" | sudo tee -a /etc/hosts

Classic usage of sudo + tee:

# Write content to a file owned by root
echo "New configuration" | sudo tee /etc/myapp.conf >/dev/null

> /dev/null suppresses tee's own output, leaving only the effect of writing to the file.

3.3 crontab: Scheduled Tasks

crontab is Linux's scheduled task tool. The syntax is crontab -e to edit, crontab -l to view, crontab -r to delete.

cron time format: minute hour day month weekday command

# Run at 3 AM every day
0 3 * * * /opt/backup.sh

# Run every 5 minutes
*/5 * * * * /opt/monitor.sh

# Every Monday at 9 AM
0 9 * * 1 /opt/weekly.sh

# At midnight on the 1st of every month
0 0 1 * * /opt/monthly.sh

Several special notations:

Several pitfalls of crontab:

  1. Different environment variables: crontab's default PATH is very limited; scripts must use absolute paths.
  2. Output is not displayed: crontab output is emailed by default, but modern servers don't have local mail — use >> /var/log/cron.log 2>&1 to redirect.
  3. Different working directory: crontab's working directory is $HOME; scripts must cd to the correct directory.

Practical advice: When writing a crontab line, specify all paths explicitly:

0 3 * * * /bin/bash /opt/backup.sh >> /var/log/backup.log 2>&1

3.4 screen / tmux: Terminal Multiplexers

screen and tmux are "terminal multiplexers" — they allow you to open multiple windows in one terminal and recover them after disconnection.

tmux is the new generation standard, with more flexible configuration than screen. If tmux is available, use tmux.

# Start a new session
tmux

# Start a named session
tmux new -s mywork

# List all sessions
tmux ls

# Reattach
tmux attach -t mywork

# Inside a session:
# Ctrl+B then c: Create new window
# Ctrl+B then n: Next window
# Ctrl+B then p: Previous window
# Ctrl+B then d: Detach session

screen has similar shortcuts to tmux (Ctrl+A instead of Ctrl+B).

Core scenario: When operating remotely, start a long-running task and use screen/tmux to protect it from being interrupted by disconnection. For example, when upgrading a service, use screen -S upgrade, and after it finishes, use screen -r upgrade to see the results.

3.5 watch: Periodic Execution

watch executes a command at regular intervals, used to observe changes.

# Check disk every 2 seconds
watch -n 2 df -h

# Check processes every second
watch -n 1 "ps aux | grep nginx"

# Highlight the changing parts
watch -d "ls -la /tmp/"

# Stop when exit code is non-zero
watch -e "curl -sf https://api.example.com/health"

watch is a good tool for dynamic monitoring, saving effort compared to manually repeating commands.

3.6 Other Tools Worth Knowing

at: One-time scheduled task. at 23:00 enters interactive mode, input commands, Ctrl+D to finish. Simpler than crontab, but not used much.

script: Record terminal sessions. script -t timing.log session.log, replay with scriptreplay after exiting.

timeout: Execute with a time limit. timeout 30 long_command, sends TERM after 30 seconds, timeout protection.

# Add a timeout to a command
timeout 10 curl -s https://slow-api.example.com

# Kill if download isn't complete after 5 seconds
timeout 5 wget https://example.com/big-file.iso

envsubst: Environment variable substitution. echo "Hello $NAME" | envsubst will replace $NAME with the environment variable's value.

# Render a configuration template
envsubst < config.template > config.conf

4. Summary

This article covered the final part of the shell common commands series — archiving, permissions, and advanced tools. The three-part series together covers 95% of daily tool scenarios.

Review of the three parts:

The first part covered file operations and text processing — ls, cp, find, cat, less, head, tail, sort, uniq, cut, paste, tr, join.

The second part covered processes, networking, and system information — ps, top, kill, nohup, ping, curl, ssh, scp, rsync, ss, dig, nc, date, free, df.

The third part (this article) covered archiving, permissions, and advanced tools — tar, gzip, chmod, chown, sudo, xargs, tee, crontab, tmux.

After reading all three, you should have a complete impression of daily shell commands, knowing when to use which and how to combine them.