macOS/Ubuntu任务队列task-spooler
1. 前言
在科研和开发中,经常会遇到这样的问题:有一批长时间运行的脚本或程序,需要一个接一个执行,而且希望方便地查看状态和日志。
macOS / Ubuntu 自带的 nohup 或后台运行 (&) 往往不够直观,也难以管理。
这时就需要:轻量级任务队列 task-spooler(ts/ tsp) 。
2. 什么是 task-spooler
task-spooler 是一个简单的本地任务队列管理工具,只需把命令交给它,它会自动排队执行,并提供日志、状态监控和优先级管理。
最大的优势是无需复杂配置,就能实现任务排队和后台管理。
3. 安装
Ubuntu 安装:
sudo apt update
sudo apt install task-spooler
macOS 安装:
brew install task-spooler
注意⚠️: macOS 用 ts 调用;Ubuntu 用 tsp 调用。
4. 基本用法
1、添加任务到队列
tsp bash job.sh
tsp python script.py
任务会自动排队,前一个任务结束后才执行下一个。
2、查看任务状态
tsp -l
状态包括:R(running 运行中)、Q(queued 排队)、D(完成)、E(错误)。
3、查看任务输出
tsp -c 0 # 查看ID为0的任务输出
tsp -o 0 # 输出文件路径
4、控制并行任务数
tsp -S 2 # 同时运行最多2个任务
5、管理任务
- 删除任务:tsp -r
- 清空队列:tsp -C
- 清除已完成任务:tsp -K
6、详细命令:
tsp -h
usage: tsp [action] [-ngfmdE] [-L <lab>] [-D <id>] [cmd...]
Env vars:
TS_SOCKET the path to the unix socket used by the ts command.
TS_MAILTO where to mail the result (on -m). Local user by default.
TS_MAXFINISHED maximum finished jobs in the queue.
TS_MAXCONN maximum number of ts connections at once.
TS_ONFINISH binary called on job end (passes jobid, error, outfile, command).
TS_ENV command called on enqueue. Its output determines the job information.
TS_SAVELIST filename which will store the list, if the server dies.
TS_SLOTS amount of jobs which can run at once, read on server start.
TMPDIR directory where to place the output files and the default socket.
Actions:
-K kill the task spooler server
-C clear the list of finished jobs
-l show the job list (default action)
-S [num] get/set the number of max simultaneous jobs of the server.
-t [id] "tail -n 10 -f" the output of the job. Last run if not specified.
-c [id] like -t, but shows all the lines. Last run if not specified.
-p [id] show the pid of the job. Last run if not specified.
-o [id] show the output file. Of last job run, if not specified.
-i [id] show job information. Of last job run, if not specified.
-s [id] show the job state. Of the last added, if not specified.
-r [id] remove a job. The last added, if not specified.
-w [id] wait for a job. The last added, if not specified.
-k [id] send SIGTERM to the job process group. The last run, if not specified.
-u [id] put that job first. The last added, if not specified.
-U <id-id> swap two jobs in the queue.
-B in case of full queue on the server, quit (2) instead of waiting.
-h show this help
-V show the program version
Options adding jobs:
-n don't store the output of the command.
-E Keep stderr apart, in a name like the output file, but adding '.e'.
-g gzip the stored output (if not -n).
-f don't fork into background.
-m send the output by e-mail (uses sendmail).
-d the job will be run only if the job before ends well
-D <id> the job will be run only if the job of given id ends well.
-L <lab> name this task with a label, to be distinguished on listing.
-N <num> number of slots required by the job (1 default).