macOS/Ubuntu任务队列task-spooler

1. 前言

在科研和开发中,经常会遇到这样的问题:有一批长时间运行的脚本或程序,需要一个接一个执行,而且希望方便地查看状态和日志。

macOS / Ubuntu 自带的 nohup 或后台运行 (&) 往往不够直观,也难以管理。

这时就需要:轻量级任务队列 task-spooler(ts/ tsp) 。

2. 什么是 task-spooler

task-spooler 是一个简单的本地任务队列管理工具,只需把命令交给它,它会自动排队执行,并提供日志、状态监控和优先级管理。

最大的优势是无需复杂配置,就能实现任务排队和后台管理。

3. 安装

Ubuntu 安装:

sudo apt update
sudo apt install task-spooler

macOS 安装:

brew install task-spooler

注意⚠️: macOS 用 ts 调用;Ubuntu 用 tsp 调用。

4. 基本用法

1、添加任务到队列

tsp bash job.sh
tsp python script.py

任务会自动排队,前一个任务结束后才执行下一个。

2、查看任务状态

tsp -l

状态包括:R(running 运行中)、Q(queued 排队)、D(完成)、E(错误)。

3、查看任务输出

tsp -c 0      # 查看ID为0的任务输出
tsp -o 0      # 输出文件路径

4、控制并行任务数

tsp -S 2      # 同时运行最多2个任务

5、管理任务

  • 删除任务:tsp -r
  • 清空队列:tsp -C
  • 清除已完成任务:tsp -K

6、详细命令:

tsp -h
usage: tsp [action] [-ngfmdE] [-L <lab>] [-D <id>] [cmd...]
Env vars:
  TS_SOCKET  the path to the unix socket used by the ts command.
  TS_MAILTO  where to mail the result (on -m). Local user by default.
  TS_MAXFINISHED  maximum finished jobs in the queue.
  TS_MAXCONN  maximum number of ts connections at once.
  TS_ONFINISH  binary called on job end (passes jobid, error, outfile, command).
  TS_ENV  command called on enqueue. Its output determines the job information.
  TS_SAVELIST  filename which will store the list, if the server dies.
  TS_SLOTS   amount of jobs which can run at once, read on server start.
  TMPDIR     directory where to place the output files and the default socket.
Actions:
  -K       kill the task spooler server
  -C       clear the list of finished jobs
  -l       show the job list (default action)
  -S [num] get/set the number of max simultaneous jobs of the server.
  -t [id]  "tail -n 10 -f" the output of the job. Last run if not specified.
  -c [id]  like -t, but shows all the lines. Last run if not specified.
  -p [id]  show the pid of the job. Last run if not specified.
  -o [id]  show the output file. Of last job run, if not specified.
  -i [id]  show job information. Of last job run, if not specified.
  -s [id]  show the job state. Of the last added, if not specified.
  -r [id]  remove a job. The last added, if not specified.
  -w [id]  wait for a job. The last added, if not specified.
  -k [id]  send SIGTERM to the job process group. The last run, if not specified.
  -u [id]  put that job first. The last added, if not specified.
  -U <id-id>  swap two jobs in the queue.
  -B       in case of full queue on the server, quit (2) instead of waiting.
  -h       show this help
  -V       show the program version
Options adding jobs:
  -n       don't store the output of the command.
  -E       Keep stderr apart, in a name like the output file, but adding '.e'.
  -g       gzip the stored output (if not -n).
  -f       don't fork into background.
  -m       send the output by e-mail (uses sendmail).
  -d       the job will be run only if the job before ends well
  -D <id>  the job will be run only if the job of given id ends well.
  -L <lab> name this task with a label, to be distinguished on listing.
  -N <num> number of slots required by the job (1 default).

5. 延伸阅读