How to Automagically Run Your Scripts
It’s cron. The answer is to use a cron job.
cron is probably my least-favorite Unix program name, except maybe grep. Such a useful command, but it is intimidating because of the name. “Just use a cronjob,” they say. Not descriptive or helpful.
Fortunately, the gist of it is very simple:
- What script do you want to run?
- When do you want it to run?
- Write it to the crontab.
- ???
- Profit.
cron is useful for scheduling repetitive tasks. If your text file is in the right place in your system, Unix will just run it at every interval specified. Each line of the file represents a “job.” Let’s walk through a simple example.
I have a program that scrapes library websites, and puts all of their events into a database. I don’t want to run it too often, let’s say once per week.
The script is located at: /home/elliott/data/scrape-events.py.
It should run: every week on Monday.
To write it to the crontab: run crontab -e, and insert 0 7 * * 1 python3 /home/elliott/data/scrape_events.py, then save it.
Okay, that was simple until we got to the last step. The crontab command is what we use to create, view, and edit cron jobs. If you run crontab -e (edit) right now, it should open an empty file.
NOTE that this command will use your default text editor. If the text editor is not working, look here for how to change the default text editor.
Wikipedia gives an excellent rundown of the syntax of the crontab:
# ┌───────────── minute (0 - 59)
# │ ┌───────────── hour (0 - 23)
# │ │ ┌───────────── day of the month (1 - 31)
# │ │ │ ┌───────────── month (1 - 12)
# │ │ │ │ ┌───────────── day of the week (0 - 6) (Sunday to Saturday;
# │ │ │ │ │ 7 is also Sunday on some systems)
# │ │ │ │ │
# │ │ │ │ │
# * * * * * python3 /home/elliott/data/scrape_events.py
A line is split into two parts: the time interval, and the script to be run.
The time should have at least one value set, and you can potentially set all of them. If you look at the job I wrote above, you can see that the time is 0 7 * * 1. This means “run at 07:00 every Monday.” If I wanted to run it every day instead, I could set it as 0 7 * * * . To run it once a month, I could instead set it as 0 7 1 * *.
Additionally, you can specify n-th intervals in two different methods. A slash runs the script at every given interval: */30 * * * will run every thirty minutes. You can also specifically list intervals: * 1,2,3 * * * will run at 01:00, 02:00, and 03:00.
Once you know your desired script, use crontab -e to edit the crontab file and save it. Unix will run cron in the background and run tasks that you defined in the crontab.
Finally, you can use crontab -l to list the contents of the crontab.
Further Thoughts
Now that you have the basics down, you can look through more intense guides. There are a few other ways you can define the schedules, but this posts covers the necessities.
However, it might be worth thinking about backing up your cron file. You may want to copy the contents to source control as backup. You may start using a different machine, or lose your current data.
Additionally, you may want to consider logging the output of your cron jobs. If they run while you are asleep, you may want to see the results after you wake up. You can use the >> bash operator to log output. For example, my python script above is actually saved as this in my crontab:
0 7 * * 1 python3 /home/elliott/data/scrape_events.py >> /home/elliott/logs/Python_log_`date +\%Y-\%m-\%d_\%H\%M`.log 2>&1