<![CDATA[Urbz Codes]]>Be sure to check out my startups:
Procuments | Tempng]]>
https://www.urbzcodes.com/https://www.urbzcodes.com/favicon.pngUrbz Codeshttps://www.urbzcodes.com/Ghost 3.12Thu, 26 Mar 2020 22:46:47 GMT60<![CDATA[Create Simple Scheduled Tasks in the Cloud with Azure’s WebJobs and TimerTriggers]]>

In two of my recent projects (Procuments and Tempng), I needed some way to utilize scheduled tasks. For Procuments I needed to send daily reminder emails and with Tempng I needed to send timecard reminders and automate payments. For both of these I found that the easiest way to accomplish

https://www.urbzcodes.com/scheduled-tasks-in-the-cloud-with-azures-webjobs-and-timertriggers/5e381175611c03001849e9c1Thu, 09 Mar 2017 15:00:00 GMT

I've consolidated what I've learned from multiple sources into this blog post.

WebJob TimerTriggers

TimerTrigger is an extension to the WebJobs SDK that allows for scheduling functions within the WebJob. You could also use Azure's Scheduler feature, but I've found the TimerTriggers to be easier to both create and manage. Plus, TimerTriggers doesn't use an additional Azure resource like Scheduler does, so no extra charges.

When first working on Procuments, I used the Azure Scheduler feature. At the time that's all I could find online about how to create scheduled tasks in the cloud. I didn't like working with Scheduler though, so when working on Tempng I did a little more research and found the TimerTriggers method.

Benefits of TimerTriggers

After implementing TimerTriggers on my Tempng WebJobs, I went back and redid my Procuments WebJobs to remove the Scheduler. I did this for multiple reasons:

  • Scheduler is not in the "New" Azure portal
  • Scheduler has a 1 hour minimum interval when using the free tier
  • TimerTriggers can go down to once a second
  • TimerTriggers use a cron expression to create the schedule
  • TimerTriggers are easier to manage, as they don't require another Azure resource (the Scheduler)

TimerTriggers are also multi-instance safe if your WebJob is spread across multiple instances. They use the WebJobs SDK Singleton that gets a lock to make sure that each timer event is only processed once. More on this later.
Note: Scheduler provides this as well.


Before you add the code, you'll need to add the WebJob Extensions NuGet package to your project: Microsoft.Azure.WebJobs.Extensions.

You'll also need to make sure that the Web App your WebJob runs on is set to "always on" in the Azure portal. This will make sure that the Azure does not stop your site if it has been idle for too long. Stopping the site means your WebJobs won't run. You also need to set the WebJob to run continuously under "How To Run" in the WebJob settings.

Adding the Code

It requires all of 4 lines to successfully get a WebJob function to run on a schedule.
In the Main() method of program.cs add the following lines

var config = new JobHostConfiguration();
var host = new JobHost(config);

You'll see that we simply added 2 new lines and overwrote the declaration of the host variable to add the newly created config.
Now update the functions themselves. Add the following to the method declaration to pass in a TimerInfo object with the cron expression dictating the desired schedule

public static void RemindersFunction([TimerTrigger("0 0 8 * * *", RunOnStartup = false)] TimerInfo timerInfo, TextWriter log) 
    //Do stuff every day at 8AM

And you're done.
Once published, this function will run every day at 8AM. You'll also notice I told it not to run on startup.

It's important to note that this will by default run at 8AM in UTC time (the exception is if you're debugging locally). To change this, add an app setting to your WebJob's App.config:

  <add key="WEBSITE_TIME_ZONE" value="Central Standard Time" />

Make sure to also add this app setting to your web app in the Azure portal.

Setting the Schedule

TimerTriggers are usually set using cron expressions, but more advanced schedules can be set using a custom schedule. We'll discuss cron expressions first as they will be used most of the time.

Cron Expressions

If you've never used cron expressions before, they are pretty easy to learn and can be a powerful tool. It may take a bit to wrap your head around it, but once you do it will be easy to create some advanced schedules.

TimerTrigger cron expressions use 6 fields instead of the usual 5:


If there's a * then it means to "run for all the x". So in our example above ( 0 0 8 * * * ) the function runs every day of the week, every month, every day of the month, the 8th hour, the 0th minute, and the 0th second. Or, 8AM every day.

Cron expressions can also use the symbols -, /, or , to add additional functionality:

  • - gives a range, so 0 0 10 * * 1-5 means to run every weekday at 10 AM
  • / makes it an interval, so 0 */15 * * * * means to run every 15 minutes. The function will run at the 0, 15, 30, and 45 minute mark every hour
  • , lets you list out specific values, so 0 0 9 1,15,31 * * means to run on the 1st, 15th, and 31st day of every month at 9AM

Here are a few more examples:

  • 0 0 * * * * - run every hour
  • 0 0 12-17 * * * - run once an hour from noon to 5PM
  • 0 0 */8 1-15 * * - run every 8 hours for the first 15 days of the month
  • 0 30/5 21 * * 0,6 - run once every 5 minutes for minutes 30-59 after 9PM on Sunday and Saturday

As you can see the schedules made using cron expressions can be very versatile. If you need help creating an expression there are websites that can help you figure out what to do. Note that many of them will not include the seconds parameter so you'll have to adjust for that.

Custom Schedules

Some schedules cannot be created using cron expressions. That's where custom schedules come into play. They can be used to make more "unusual" schedules. For instance, say I want to schedule a function for Tuesdays at 9AM and 5PM, and Thursdays at noon. Cron expressions don't allow for this, but custom schedules do. To do so we'll have to add a new class to our code:

public class MyWeeklySchedule : WeeklySchedule
    public MyWeeklySchedule()
        // Every Tuesday at 9AM and 5PM
        Add(DayOfWeek.Tuesday, new TimeSpan(9, 0, 0));
        Add(DayOfWeek.Tuesday, new TimeSpan(17, 0, 0));

        // Every Thursday at 12PM
        Add(DayOfWeek.Thursday, new TimeSpan(12, 0, 0));

The WeeklySchedule class is a part of the Microsoft.Azure.WebJobs.Extensions.Timers namespace.

Then we add the schedule in the function's method:

public static void RemindersFunction([TimerTrigger(typeof(MyWeeklySchedule))] TimerInfo timerInfo, TextWriter log) 
    //Do stuff

There is also a DailySchedule class that allows for multiple times per day:

public class MyDailySchedule : DailySchedule
    public MyDailySchedule() :
        base("7:30:00", "12:00:00", "17:30:00")
    { }



If you want to test your cron jobs locally, there are a few things you should do to help facilitate this. After creating the config in your program.cs, add the following:

//If you want to set it manually. 15 seconds is the lowest it can go
//config.Singleton.ListenerLockPeriod = new TimeSpan(0, 0, 15);

The ListenerLockPeriod setting will tell the Singleton how long it can hold on to the lock. This is part of the multi-instance safety that WebJobs provides. The lock is a Storage Blob created automatically in the attached storage container of your WebJob. If an instance is already holding onto the lock then a new instance of your function will not be started.

This can cause problems when testing locally since you will be stopping and starting the solution frequently. Once stopped ungracefully (closing the console window, etc), the lock is still held on to for the ListenerLockPeriod (default is 60 seconds). This means that your function will not be able to get a lock, and will not start. Telling the config to UseDevelopmentSettings() will set this to 15 seconds and set a number of magic settings that will help while testing locally. This allows you to restart the app more quickly while testing.

This also means that if your function is running in the cloud on Azure you won't be able to run locally, since the instance in the cloud still has the blob locked. You'll have to either use a different storage account or temporarily stop the job on Azure.


You can find some more in-depth reading on TimerTriggers here and here. I've covered the important parts, but more reading can never hurt.

I've found working with TimerTriggers to be exceptionally easy and have yet to run into any issues with them. The ease of implementation makes them a no-brainer when looking for a method to create scheduled tasks in the cloud.

<![CDATA[C#: LINQ Proximity Query Performance with the SQL Geography Data Type]]>

In one of my current projects, Tempng, I needed to be able to find the closest users from the location of a customer on a per request basis. A customer will make a request, which will kick off an Azure WebJob that does the actual work of finding the closest

https://www.urbzcodes.com/c-linq-proximity-query-performance/5e381175611c03001849e9bfMon, 20 Jun 2016 21:45:55 GMT

In one of my current projects, Tempng, I needed to be able to find the closest users from the location of a customer on a per request basis. A customer will make a request, which will kick off an Azure WebJob that does the actual work of finding the closest users and processing the results. The best way I've found to do this is to use the SQL Server geography data type.

The Background

Both the customers and users have a set location (their address) that is stored in our database, so that makes it a little easier. Since all of the locations in my project are static, I considered calculating the distance between every customer and every user and storing that in a table, but that would get out of hand very quickly. So I went with the geography data type. The main reason I chose it was the geospatial proximity calculations it provides. Using LINQ, I can easily query based on the distances between locations.
For instance, say I wanted to find the users less than 10 miles from my customer Bob, the query is as simple as:

var users = from g in context.GeoTests
    let distance = g.Location.Distance(bobsLocation)
    where distance < 10 * 1609.344    //Makes it miles. 1609.344 is meters in a mile
    orderby distance
    select new { UserId = g.UserId, DistanceInMiles = (distance / 1609.344) };

Note that the distance value is calculated in meters, so we divide by the number of meters in a mile (1609.344) to calculate miles.

But Does it Scale?

Testing at a small scale worked very well, so I was curious about the scaling performance of such queries. I decided to do a few tests. Here's the table structure I used:
GeoTest Table

Creation Performance

First I wanted to test the performance of creating and inserting the geography values into my database. I tested this by populating the tables with random US locations. I found the longitude and latitude bounds by very scientifically clicking what I thought looked the northern, southern, western, and eastern most points in Google maps.

  • Northern most
  • 49.259766, -95.616505
  • Southern most
  • 26.280313, -97.530646
  • Western most
  • 40.497140, -124.307168
  • Eastern most
  • 44.774586, -67.034416

I used the following code to populate the table from my laptop into an Azure SQL Database.

using (var context = new GeoTestEntities())
    for (var i = 0; i < RECORDS_TO_CREATE; i++)
        var lon = RandomNumberBetween(-124, -67);
        var lat = RandomNumberBetween(26, 50);
        var geo = new DAL.GeoTest()
            Lon = lon,
            Lat = lat,
            Location = DbGeography.FromText("POINT(" + lon + " " + lat + ")", 4326)

You'll need to add the System.Data.Entity.Spatial namespace to get the DbGeography class. The RandomNumberBetween() method returns a decimal. The value 4326 is the coordinate system Id, and is the most common system used in mapping and geolocation.

I ran each test 3 times. Here are the averages and a graph showing their relationship:

Number of RecordsTime in Seconds
![Populate Table Results](/content/images/2016/06/create-results-5.PNG) So creating the `geography` records is more or less linear. It's pretty slow as far as computing goes, but we'll only be creating 1 of these at a time when the user signs up. On top of that, it runs through a WebJob, so the user won't notice any slowdown at all when entering in their address.
Proximity Query Performance

The real test is querying all this data. I wanted to see how long it would take to perform a proximity query when dealing with more than the 10 locations I originally tested with. If the query doesn't scale well, then we'll run into a huge problem later on down the road.

I created 5 different tables and populated them with the same script from above. Each one has a different number of records. Then I ran a query to find all locations (our users) within 100 miles of Dallas (our customer). I ran it 3 different times over each of the 5 tables and then took the average:

var dallas = DbGeography.FromText("POINT(-96.796988 32.776664)", 4326);
var locations = from g in context.GeoTests
    let distance = g.Location.Distance(dallas)
    where distance < 100 * 1609.344 //Within 100 miles of Dallas
    orderby distance
    select new { Id = g.Id, DistanceInMiles = (distance / 1609.344)
Number of RecordsTime in Seconds# Results
![100 Mile Query Results](/content/images/2016/06/query-results1-1.PNG) You can see that querying the `geography` type is also linear. In Big O notation it's *O(n)*. Nice! It averages to about 2500 records per second.

I also wanted to see if the distance parameter I was using made a difference, so I ran the tests again checking for locations within 15 miles instead of 100:

Number of RecordsTime in Seconds# Results
![15 Mile Query Results](/content/images/2016/06/query-results2-2.PNG) This took a little less time, averaging 2700 records per second. So while the distance parameter does have an affect, it's not really that noticeable for what I'm doing in my project. Most if not all of my proximity searches will be less than 100 miles.

The Takeaway

Searching through 20,000 user's locations takes less than 10 seconds, and considering that these queries will all be run out of band in WebJobs, I am perfectly happy with the scaling abilities of querying the geography data.
I could make the query even quicker by restricting the query further by only looking at users in a specific zip code or city or even state, but with the performance I'm seeing that won't be necessary until we get to many tens of thousands of users.