Introduction
One way schedule jobs using Quartz.NET that are dependent upon each other and require specific sequential execution.
https://www.quartz-scheduler.net/
Background
There are some tutorials that provide ways of executing jobs sequentially however they don't make clear the notion of Durable vrs Non-Durable in Quartz. Quartz defines jobs that are non-durable having an life span tied to the existence of it's triggers. In the case of how the below sequential execution works, the only trigger exists on first exeuction so the trouble I ran into was knowing how to define the other dependent jobs as Durable. This allows the implementation of the job via Windows Service to execute all jobs sequentially instead of just the first scheduled job.
A few things I hope to address that were issues with my learning Quartz are
- Setting up jobs to execute beyond the first execution
- After the last sequentially executed job runs, run a unspecified number of jobs concurrently.
The situation I'll present here is comprised of 3 steps. 1) You need to charge a bunch of pending payments via a scheduled job. 2) Then once all payments are charged, you generate a report of all payments that were charged and upload it somewhere. 3) Finally, once the payments have been charged, batch report of payments successfully charged has been generated, send email and text messages updating the customers that their cards have been charged.
I'm not actually going to include code to charge payments, generate reports, or email/text others, more the situation of how 2) can only run once 1) has completed and then fire off multiple jobs in 3) that only depend on 2) having completed.
Using the code
The portions of this example can be broken down into
- Job classes
- JobListener classes
- The scheduler itself
Quartz.net can be implemented using their server/enterprise service functionality but for the purpose of this example it will just be a console app. You can easily apply the same code to a windows service also.
Jobs
Declare what jobs you may need to execute in your application. For our purposes. Going to create a ChargePaymentsJob, BatchFileUploadJob, TextMessageJob, and EmailJob. ChargePayments must run first, then BatchFileUploadJob. Once BatchFileUploadJob has completed, we can let Quartz.NET fire off the TextMessageJob and EmailJob at the same time as neither of these jobs depend on the other in order to complete.
All of your jobs need to inherit from IJob interface in order to let the scheduler know that it is something that the scheduler should execute.
public class EmailJob : IJob
{
public void Execute(IJobExecutionContext context)
{
for (int i = 0; i < 3; i++)
{
Console.WriteLine("Email sent to customer {0} payment was charged successfully", i);
}
}
}
public class TextMessageJob : IJob
{
public void Execute(IJobExecutionContext context)
{
for (int i = 0; i < 3; i++)
{
Console.WriteLine("Text mesasge sent to customer {0} payment was charged successfully", i);
}
}
}
public class BatchFileUploadJob : IJob
{
public void Execute(IJobExecutionContext context)
{
Console.WriteLine("Report of payments processed successfully generated");
}
}
public class ChargePaymentsJob : IJob
{
public void Execute(IJobExecutionContext context)
{
for (int i = 0; i < 5; i++)
{
Console.WriteLine("Charging Customer {0}s payment", i);
}
}
}
JobListeners
JobListeners are classes that respond to events that occur from the scheduler/jobs that are executed. Quartz has a notion of TriggerListeners as well but I won't get into that here as using TriggerListeners isn't necessary for this example.
For this example we'll have a DependentJobListener and a NonDepedentJobListener. You could Apply the same listener for both types of jobs, but you may have a case where different things must happen for DependentJobs vrs NonDependent jobs so it may be better to separate this out altogether.
Your job listener must inherit from IJobListener interface which provides 3 methods, JobToBeVetoed, JobToBeExecuted, JobWasExecuted. The one that we care about in this example is JobWasExecuted. However, JobToBeVetoed is only called if the job itself was vetoed from a TriggerListener and JobToBeExecuted is only called as long as the job hasn't been vetoed. The notion of veto here means whether or not the job has been rejected based on the trigger conditions or any customer TriggerListeners that were implemented.
In the JobWasExecuted method, the first thing we do is handle any errors that may occur by checking if the jobException parameter is null or not.
var jobname = context.JobDetail.Key.Name;
if (jobException != null)
{
Console.WriteLine("Job {0} exploded, unable to complete the tasks it required", jobname);
return;
}
Then if all is well, once the job BatchFileUploadJob executed, we'll schedule all the other jobs that depend on ChargePaymentsJob and BatchFileUploadJob to have ran but don't depend on each other to run by adding a condition to say "If currently executed job name equals the last dependent job" which would then provide the trigger needed to schedule all futher non-dependent jobs for immediate execution.
if (jobname == "BatchFileUploadJob")
{
}
After that, Instead of just hard coding these jobs, I've provided a way of implementing these jobs via reflection. The intent here is that you could then move these things to a table in your database and as your requirements change over time, you could simply add a NonDependent Job to your table and then loop over it as seen below in order to schedule it for immediate execution after all Dependent Jobs have ran.
The important thing here to note is you need to store the fully qualified class name of your job along with the unique name of the job to be executed.
var nonDependentJobsFromDataStore = new Dictionary<string, string>();
nonDependentJobsFromDataStore.Add("RemindersJob", "QuartzDependentJobScheduling.TextMessageJob");
nonDependentJobsFromDataStore.Add("EmailJob", "QuartzDependentJobScheduling.EmailJob");
var jobs = new List<Tuple<JobKey, string>>();
foreach (var ndjob in nonDependentJobsFromDataStore)
{
jobs.Add(Tuple.Create(new JobKey(ndjob.Key, "NonDependentJob"), ndjob.Value));
}
Once all jobs have been loaded from an external source (or hard coded) you need to massage that data into Jobs and Triggers that the scheduler can then execute.
Creating a list of JobDetails will allow you to then attach a trigger to those jobs in the second loop. You have to create new triggers for each job you are adding (whether you hard code or not) otherwise Quartz.NET throws an exception. Forgot the exact exception but it basically says you can't reuse the same trigger for multiple jobs, each job must have its own trigger.
var jobDetails = new List<IJobDetail>();
foreach (var item in jobs)
{
IJobDetail job = JobBuilder.Create(Type.GetType(item.Item2))
.WithIdentity(item.Item1)
.Build();
jobDetails.Add(job);
}
foreach (var detail in jobDetails)
{
ITrigger trigger = TriggerBuilder.Create()
.StartNow()
.Build();
context.Scheduler.ScheduleJob(detail, trigger);
}
Full Job Listener classes
public class DependentJobListener : IJobListener
{
public void JobToBeExecuted(IJobExecutionContext context)
{
}
public void JobExecutionVetoed(IJobExecutionContext context)
{
}
public void JobWasExecuted(IJobExecutionContext context, JobExecutionException jobException)
{
var jobname = context.JobDetail.Key.Name;
if (jobException != null)
{
Console.WriteLine("Job {0} exploded, unable to complete the tasks it required", jobname);
return;
}
if (jobname == "BatchFileUploadJob")
{
var nonDependentJobsFromDataStore = new Dictionary<string, string>();
nonDependentJobsFromDataStore.Add("RemindersJob", "QuartzDependentJobScheduling.TextMessageJob");
nonDependentJobsFromDataStore.Add("EmailJob", "QuartzDependentJobScheduling.EmailJob");
var jobs = new List<Tuple<JobKey, string>>();
foreach (var ndjob in nonDependentJobsFromDataStore)
{
jobs.Add(Tuple.Create(new JobKey(ndjob.Key, "NonDependentJob"), ndjob.Value));
}
var jobDetails = new List<IJobDetail>();
foreach (var item in jobs)
{
IJobDetail job = JobBuilder.Create(Type.GetType(item.Item2))
.WithIdentity(item.Item1)
.Build();
jobDetails.Add(job);
}
foreach (var detail in jobDetails)
{
ITrigger trigger = TriggerBuilder.Create()
.StartNow()
.Build();
context.Scheduler.ScheduleJob(detail, trigger);
}
}
}
public string Name
{
get { return "DependentJobListener"; }
}
}
public class NonDependentJobListener : IJobListener
{
public void JobToBeExecuted(IJobExecutionContext context)
{
}
public void JobExecutionVetoed(IJobExecutionContext context)
{
}
public void JobWasExecuted(IJobExecutionContext context, JobExecutionException jobException)
{
var jobname = context.JobDetail.Key.Name;
if (jobException != null)
{
Console.WriteLine("NonDependent Job {0} exploded, unable to complete the tasks it required", jobname);
return;
}
Console.WriteLine("NonDependent Job - {0} executed successfully", jobname);
}
public string Name
{
get { return "NonDependentJobListener"; }
}
}
Scheduler
First thing to do is to initialize the scheduler and then declare the jobs that need to be ran. The thing that tripped me up here is calling the .StoreDurably(true) method when declaring each job. If you don't do that here, all jobs that aren't directly called via .ScheduleJob method will only run once.
You could also call your dependent jobs from a database as well and declare the jobs via reflection here too. But for the ease of showing job chaining, all jobs are hard coded.
ISchedulerFactory schedFactory = new StdSchedulerFactory();
IScheduler scheduler = schedFactory.GetScheduler();
scheduler.Start();
var chargePaymentsJobKey = JobKey.Create("ChargePaymentsJob", "DependentJob");
var batchFileUploadJobKey = JobKey.Create("BatchFileUploadJob", "DependentJob");
IJobDetail chargePaymentsJob = JobBuilder.Create<ChargePaymentsJob>()
.WithIdentity(chargePaymentsJobKey)
.Build();
IJobDetail batchFileUploadJob = JobBuilder.Create<BatchFileUploadJob>()
.WithIdentity(batchFileUploadJobKey)
.StoreDurably(true)
.Build();
This example cron string is setup to run the jobs every minute. You could change it to 0/5 for every five minutes or follow the provided tutorial and schedule the jobs based on your needs.
ITrigger dependentJobTrigger = TriggerBuilder.Create()
.WithCronSchedule("0 0/1 * * * ?")
.Build();
After you've created your jobs, you then should add your job listeners, The important thing to note here is that if you choose to use multiple job listeners, that you assign each job listener to the correct corresponding group of the jobs that you create. In this case, our two groups are DependentJob and NonDependentJob.
scheduler.ListenerManager.AddJobListener(new DependentJobListener(), GroupMatcher<JobKey>.GroupEquals("DependentJob"));
scheduler.ListenerManager.AddJobListener(new NonDependentJobListener(), GroupMatcher<JobKey>.GroupEquals("NonDependentJob"));
The last step is to then chain and schedule your jobs. In this simple example, there is only one level of Job chaining going on so I've included a commented out line to indicate how you would chain additional jobs.
JobChainingJobListener listener = new JobChainingJobListener("DependentJobChain");
listener.AddJobChainLink(chargePaymentsJobKey, batchFileUploadJobKey);
scheduler.ListenerManager.AddJobListener(listener, GroupMatcher<JobKey>.GroupEquals("DependentJob"));
In the commented out line, nextDependentJobKey would be the third (and final) DependentJob to be executed. The format of multiple dependent job chaining would go something like:
listener.AddJobChainLink(firstJobKey, secondJobKey);
listener.AddJobChainLink(secondJobKey, thirdJobKey);
listener.AddJobChainLink(thirdJobKey, fourthJobKey);
Which would follow the above mentioned jobKey variables when chaining them all together. Once you've chained all jobs necessary, you then add the listener variable to the ListenerManager for the scheduler via the AddJobListener method and indicate what group this applys to as well.
scheduler.ListenerManager.AddJobListener(listener, GroupMatcher<JobKey>.GroupEquals("DependentJob"));
Finally, you then schedule the first job, chargePaymentsJob, with the first and only tigger, dependentJobTrigger. Then for every job after that, you call the .AddJob method since the execution of each additional job is dependent upon the first scheduled job.
scheduler.ScheduleJob(chargePaymentsJob, dependentJobTrigger);
scheduler.AddJob(batchFileUploadJob, false, false);
History
8/18/2017 - Initial Article written
I'm a software developer and enjoy .net, angular, mvc, and many other different languages.
Submit an article or tip
