How to run a Linux script every few seconds under cron

From Computer Tyme Support Wiki

(Difference between revisions)
Jump to: navigation, search
(Even more precise with microsecond resolution)
(Overview)
Line 1: Line 1:
= Overview =
= Overview =
-
Did you ever want to run a program every few seconds under a linux, unix, bsd or osx cron script? Here's an elegant script that does just that. You can get 1,2,3,4,5,6,10,12,15,20,30 second resolution. Just run it once a minute under crond and it just works.
+
Did you ever want to run a program every few seconds, or even fractions of a second, under a linux, unix, bsd or osx cron script? Here's an elegant script that does just that.  
Features:
Features:
* Run once per minute under cron
* Run once per minute under cron
* Launches multiple programs in parallel
* Launches multiple programs in parallel
-
* Multiple time periods supported simultaneously
+
* Multiple time periods supported simultaneously just by the directory name.
-
* Use of timeout kills programs that don't terminate within delay time
+
* It just works
* It just works
* Simple, elegant, free
* Simple, elegant, free
-
 
-
I'm now posting two versions of the program. The first version is simpler and is good for running as many as 30 times a minute. There are 2 issues however. crond as it turns out isn't real precise and it starts programs 1 to 1.5 seconds after the minute. Also - since the program call itself over and over during the minute there is some drift. There's enough drift so you can't really use it to run something every second. But every 2 seconds or greater is fine. Also - if you don't need to make sure the programs are killed before the next interval you can get rid of the timeout code.
 
Even though this software is free if you find it really useful and you want to reward/encourage me you can email me an Amazon Gift Certificate to marc@perkel.com.
Even though this software is free if you find it really useful and you want to reward/encourage me you can email me an Amazon Gift Certificate to marc@perkel.com.
-
== Simple Example ==
 
-
 
-
<pre>
 
-
#! /bin/sh
 
-
 
-
# Run all programs in a directory in parallel
 
-
# Usage: run-parallel directory delay
 
-
# Copyright 2013 by Marc Perkel
 
-
# docs at http://wiki.junkemailfilter.com/index.php/How_to_run_a_Linux_script_every_few_seconds_under_cron"
 
-
# Free to use with attribution
 
-
 
-
if [ $# -eq 0 ]
 
-
then
 
-
    echo
 
-
    echo "run-parallel by Marc Perkel"
 
-
    echo
 
-
    echo "This program is used to run all programs in a directory in parallel"
 
-
    echo "or to rerun them every X seconds for one minute."
 
-
    echo "Think of this program as cron with seconds resolution."
 
-
    echo
 
-
    echo "Usage: run-parallel [directory] [delay]"
 
-
    echo
 
-
    echo "Examples:"
 
-
    echo "  run-parallel /etc/cron.20sec 20"
 
-
    echo "  run-parallel 20"
 
-
    echo "  # Runs all executable files in /etc/cron.20sec every 20 seconds or 3 times a minute."
 
-
    echo
 
-
    echo "If delay parameter is missing it runs everything once and exits."
 
-
    echo "If only delay is passed then the directory /etc/cron.[delay]sec is assumed."
 
-
    echo
 
-
    echo 'If "cronsec" is passed then it runs all of these delays 2 3 4 5 6 10 12 15 20 30'
 
-
    echo "resulting in 30 20 15 12 10 6 5 4 3 2 executions per minute."
 
-
    echo
 
-
    exit
 
-
fi
 
-
 
-
# If "cronsec" is passed as a parameter then run all the delays in parallel
 
-
 
-
if [ $1 = cronsec ]
 
-
then
 
-
    for interval in 2 3 4 5 6 10 12 15 20 30
 
-
    do
 
-
      $0 $interval &
 
-
    done
 
-
    exit
 
-
fi
 
-
 
-
# Set the directory to first parameter and delay to second parameter
 
-
 
-
dir=$1
 
-
delay=$2
 
-
 
-
# If only parameter is 2,3,4,5,6,10,12,15,20,30 then automatically calculate
 
-
# the standard directory name /etc/cron.[delay]sec
 
-
 
-
if [[ "$1" =~ ^(2|3|4|5|6|10|12|15|20|30)$ ]]
 
-
then
 
-
    dir="/etc/cron.$1sec"
 
-
    delay=$1
 
-
fi
 
-
 
-
# Exit if directory doesn't exist or has no files
 
-
 
-
if [ ! "$(ls -A $dir/ 2> /dev/null)" ]
 
-
then
 
-
    exit
 
-
fi
 
-
 
 
-
# Sleep if $counter is set
 
-
 
-
if [ ! -z $counter ]
 
-
then
 
-
    sleep $delay
 
-
fi
 
-
 
-
# Run all the programs in the directory in parallel
 
-
# Use of timeout ensures that the processes are killed if they run too long
 
-
 
-
for program in $dir/* ; do
 
-
    if [ -x $program ]
 
-
    then
 
-
      if [ 0$delay -gt 0 ]
 
-
      then
 
-
          timeout $delay $program &> /dev/null &
 
-
      else
 
-
          $program &> /dev/null &
 
-
      fi
 
-
    fi
 
-
done
 
-
 
-
# If delay not set then we're done
 
-
 
-
if [ -z $delay ]
 
-
then
 
-
    exit
 
-
fi
 
-
 
-
# Add delay to counter
 
-
 
-
counter=$(( $counter + $delay ))
 
-
 
-
# If minute is not up - call self recursively
 
-
 
-
if [ $counter -lt 60 ]
 
-
then
 
-
    . $0 $dir $delay &
 
-
fi
 
-
 
-
# Otherwise we're done
 
-
</pre>
 
-
 
-
You can then create the directories needed to put your programs in that you want to run. You don't have to create the directories that you aren't going to use.
 
-
 
-
mkdir /etc/cron.2sec
 
-
mkdir /etc/cron.3sec
 
-
mkdir /etc/cron.4sec
 
-
mkdir /etc/cron.5sec
 
-
mkdir /etc/cron.6sec
 
-
mkdir /etc/cron.10sec
 
-
mkdir /etc/cron.12sec
 
-
mkdir /etc/cron.15sec
 
-
mkdir /etc/cron.20sec
 
-
mkdir /etc/cron.30sec
 
-
 
-
To run every minute you can edit your /etc/crontab file and add:
 
-
 
-
* * * * * root /usr/local/sbin/run-parallel cronsec
 
-
 
-
Keep in mind that the programs that you are trying need to be written to complete in the allotted time or they will be killed before completing. Also keep in mind that on a heavily loaded system that on smaller intervals there could be some time drift over the 1 minute period with unknown results. This is especially true of the 1 second interval.
 
-
 
-
Have a comment? You can email me at support@junkemailfilter.com
 
-
 
-
== More Precise Implementation ==
 
-
 
-
Maybe I'm too picky but I want it to start right on the minute and have the intervals not be subject to drift/loading errors. So here is a slightly more complex version. The only drawback to this version is that it will take at least a full minute to start and maybe up to 2 minutes. That's because when it is launched it waits out the first minute to start processes right at the top of the minute instead of being late.
 
-
 
-
The usleep command is required for microsecond resolution sleep times.
 
-
 
-
This implementation starts on time and has 1 second resolution. Accuracy on my servers seem to be about 15 ms or better.
 
-
 
-
<pre>
 
-
#! /bin/sh
 
-
 
-
# Run all programs in a directory in parallel
 
-
# Usage: run-parallel directory delay
 
-
# Copyright 2013 by Marc Perkel
 
-
# docs at http://wiki.junkemailfilter.com/index.php/How_to_run_a_Linux_script_every_few_seconds_under_cron"
 
-
# Free to use with attribution
 
-
 
-
if [ $# -eq 0 ]
 
-
then
 
-
  echo
 
-
  echo "run-parallel by Marc Perkel"
 
-
  echo
 
-
  echo "This program is used to run all programs in a directory in parallel"
 
-
  echo "or to rerun them every X seconds for one minute."
 
-
  echo "Think of this program as cron with seconds resolution."
 
-
  echo
 
-
  echo "Usage: run-parallel [directory] [delay]"
 
-
  echo
 
-
  echo "Examples:"
 
-
  echo "  run-parallel /etc/cron.20sec 20"
 
-
  echo "  run-parallel 20"
 
-
  echo "  # Runs all executable files in /etc/cron.20sec every 20 seconds or 3 times a minute."
 
-
  echo
 
-
  echo "If delay parameter is missing it runs everything once and exits."
 
-
  echo "If only delay is passed then the directory /etc/cron.[delay]sec is assumed."
 
-
  echo
 
-
  echo 'If "cronsec" is passed then it runs all of these delays 1 2 3 4 5 6 10 12 15 20 30'
 
-
  echo "resulting in 60 30 20 15 12 10 6 5 4 3 2 executions per minute."
 
-
  echo
 
-
  exit
 
-
fi
 
-
 
-
# Runs like the sleep command but compensates for drift to start on time
 
-
 
-
function syncsleep ()
 
-
{
 
-
  usleep $(( $1000000 - (10#$(date +%N) / 1000) ))
 
-
}
 
-
 
-
# If "cronsec" is passed as a parameter then run all the delays in parallel
 
-
 
-
if [ $1 = cronsec ]
 
-
then
 
-
 
-
  # Wait till start of next minute
 
-
 
-
  syncsleep $(( 60 - 10#$(date +%S) ))
 
-
 
-
  for interval in 1 2 3 4 5 6 10 12 15 20 30
 
-
  do
 
-
      $0 $interval &
 
-
  done
 
-
 
-
  exit
 
-
fi
 
-
 
-
# Set the directory to first parameter and delay to second parameter
 
-
 
-
dir=$1
 
-
delay=$2
 
-
 
-
# If only parameter is 1,2,3,4,5,6,10,12,15,20,30 then automatically calculate
 
-
# the standard directory name /etc/cron.[delay]sec
 
-
 
-
if [[ "$1" =~ ^(1|2|3|4|5|6|10|12|15|20|30)$ ]]
 
-
then
 
-
  dir="/etc/cron.$1sec"
 
-
  delay=$1
 
-
fi
 
-
 
-
# Exit if directory doesn't exist or has no files
 
-
 
-
if [ ! "$(ls -A $dir/ 2> /dev/null)" ]
 
-
then
 
-
  exit
 
-
fi
 
-
 
-
# Sleep if $counter is set
 
-
 
-
if [ ! -z $counter ] 
 
-
then
 
-
  syncsleep $delay
 
-
fi
 
-
 
-
# Run all the programs in the directory in parallel
 
-
# Use of timeout ensures that the processes are killed if they run too long
 
-
 
-
for program in $dir/* ; do
 
-
  if [ -x $program ]
 
-
  then
 
-
      if [ 0$delay -gt 0 ]
 
-
      then
 
-
        timeout $delay $program &> /dev/null &
 
-
      else
 
-
        $program &> /dev/null &
 
-
      fi
 
-
  fi
 
-
done
 
-
 
-
# If delay not set then we're done
 
-
 
-
if [ -z $delay ]
 
-
then
 
-
  exit
 
-
fi
 
-
 
-
# Add delay to counter
 
-
 
-
counter=$(( $counter + $delay ))
 
-
 
-
# If minute is not up - call self recursively
 
-
 
-
if [ $counter -lt 60 ]
 
-
then
 
-
  . $0 $dir $delay &
 
-
fi
 
-
 
-
# Otherwise we're done
 
-
</pre>
 
== Even more precise with microsecond resolution ==
== Even more precise with microsecond resolution ==
-
 
+
Tthe existence of the directory creates the schedule. This version supports microsecond resolution assuming your computer can handle it. It is the most accurate on start time and actually us a more simple implementation. Also the number of executions per minute does not have to to evenly divisible by 60. If you want to run something 17 times a minute - no problem. You can also run faster than once a second.
-
Here I changed the name of the program because the way it works is different. here the existence of the directory creates the schedule. This version supports microsecond resolution assuming your computer can handle it. It is the most accurate on start time and actually us a more simple implementation. Also the number of executions per minute does not have to to evenly divisible by 60. If you want to run something 17 times a minute - no problem.
+
Even though this has microsecond resolution probably 2400 per minute is as fast as you can practically go.
Even though this has microsecond resolution probably 2400 per minute is as fast as you can practically go.

Revision as of 02:32, 17 February 2014

Overview

Did you ever want to run a program every few seconds, or even fractions of a second, under a linux, unix, bsd or osx cron script? Here's an elegant script that does just that.

Features:

  • Run once per minute under cron
  • Launches multiple programs in parallel
  • Multiple time periods supported simultaneously just by the directory name.
  • It just works
  • Simple, elegant, free

Even though this software is free if you find it really useful and you want to reward/encourage me you can email me an Amazon Gift Certificate to marc@perkel.com.


Even more precise with microsecond resolution

Tthe existence of the directory creates the schedule. This version supports microsecond resolution assuming your computer can handle it. It is the most accurate on start time and actually us a more simple implementation. Also the number of executions per minute does not have to to evenly divisible by 60. If you want to run something 17 times a minute - no problem. You can also run faster than once a second.

Even though this has microsecond resolution probably 2400 per minute is as fast as you can practically go.

Keep in mind that this program takes up to 2 minutes to start.

#! /bin/sh

# Microsecond Cron
# Usage: cron-ms start
# Copyright 2014 by Marc Perkel
# docs at http://wiki.junkemailfilter.com/index.php/How_to_run_a_Linux_script_every_few_seconds_under_cron"
# Free to use with attribution

basedir=/etc/cron-ms

if [ $# -eq 0 ]
then
   echo
   echo "cron-ms by Marc Perkel"
   echo
   echo "This program is used to run all programs in a directory in parallel every X times per minute."
   echo "Think of this program as cron with microseconds resolution."
   echo
   echo "Usage: cron-ms start"
   echo
   echo "The scheduling is done by creating directories with the number of"
   echo "executions per minute as part of the directory name."
   echo
   echo "Examples:"
   echo "  /etc/cron-ms/7      # Executes everything in that directory  7 times a minute"
   echo "  /etc/cron-ms/30     # Executes everything in that directory 30 times a minute"
   echo "  /etc/cron-ms/600    # Executes everything in that directory 10 times a second"
   echo "  /etc/cron-ms/2400   # Executes everything in that directory 40 times a second"
   echo
   exit
fi

# If "start" is passed as a parameter then run all the loops in parallel
# The number of the directory is the number of executions per minute
# Since cron isn't accurate we need to start at top of next minute

if [ $1 = start ]
then
   for dir in $basedir/* ; do
      $0 ${dir##*/} 60000000 &
   done
   exit
fi

# Loops per minute and the next interval are passed on the command line with each loop

loops=$1
next_interval=$2

# Sleeps until a specific part of a minute with microsecond resolution. 60000000 is full minute

usleep $(( $next_interval - 10#$(date +%S%N) / 1000 ))

# Run all the programs in the directory in parallel

for program in $basedir/$loops/* ; do
   if [ -x $program ] 
   then
      $program &> /dev/null &
   fi
done

# Calculate next_interval

next_interval=$(($next_interval % 60000000 + (60000000 / $loops) ))

# If minute is not up - call self recursively

if [ $next_interval -lt $(( 60000000 / $loops * $loops)) ]
then
   . $0 $loops $next_interval &
fi

# Otherwise we're done

Launch from cron every minute. Edit your /etc/crontab file and add:

* * * * * root /usr/local/sbin/cron-ms start

Kind of amazing that you can do all this in less than 50 lines of code.

Personal tools