Requirements
I was recently introduced to the Nintex Workflow 2010 product several months ago, and during construction of one of my workflows for a client, I found myself needing recursion. Why, you ask? Well, this problem always seems to surface whenever a hierarchy is involved, such as folders in a file system that have subfolders. In my case, I had a list in SharePoint that contained a template of tasks that were to occur in a certain order, and each task had a relationship to another task. For example, “Set up utilities” must occur 30 days before from “Move in to my new house”. What I knew ahead of time was only “Move-in Date”, and I wanted my workflow to basically walk through the hierarchy of related tasks and calculate dates for all of my tasks.
If I were coding this in a programming language, I’d just create a recursive method that called itself with the base date and passed that to all related tasks so they could do their calculations. However, in order to hand this calculation procedure over to a client that had no coders on staff, Nintex Workflow was the chosen vehicle for delivery. When I first used Nintex, I didn’t easily spot a construct that would allow me to implement recursion. Sure, workflows can call other workflows, but the number of simultaneous executing workflows could get out of hand causing SharePoint to rain on that parade rather quickly.
Prerequisite Knowledge
It’s helpful at this point to step back a bit and think about what we’re trying to do. The feature I want in Nintex is basically the ability to call a function (method is more of an object oriented programming label), but Nintex doesn’t provide the concept of a function. But what is a function, and how does recursion within functions normally work in a standard programming environment you ask? (you ask a lot of questions, but that’s a good thing)
Computer Science 101 and 201
When I was in college getting my bachelor’s degree in Computer Science, there were a couple courses I took that proved helpful to me when determining how to set this up. The first is Assembly Programming 101, where one learns exactly what happens behind the scenes when a function is called. The short answer is to use a data structure called a stack, which I learned in another class, Data Structures 101. In assembly, when you want to call a function that requires a parameter, you “push” the parameter (using a register to store the value) onto the stack first. Then, once inside the function, you “pop” the value off the stack to use it. So, the next time you’re writing methods in C#, just think about all those stack operations going on behind the scenes on your behalf. :)
Helpful animation explaining stacks
So, now that you understand how a function call might work with a stack, how does recursion work? Instead of scaring you away with terms like “depth-first preorder tree traversal” (also from Data Structures 101), just know that we’ll need to keep a running list of tasks that have related tasks, and we’ll use a stack for that.
Here’s the algorithm:
- Every time you find a task (parent) that has related tasks (children), push the parent to the stack.
- Loop over the stack, pop an item off, and get children.
- Repeat step 1 and 2 until the list is empty.
Helpful animations of tree traversals
See, easy as pie, right?
Nintex Workflow 2010 Tools
Now that we’re familiar with the building blocks of what we’ll need, let’s take a look of what Nintex does provide us with to see if we can implement our algorithm.
A stack can be implemented in Nintex as a collection variable. Collection variables in Nintex are ordered 1-dimensional arrays (for the programmers out there). You cannot store something like an object with properties in a collection variable. If you need a collection of objects, you’d just use multiple collections and load them in the same order (they remain ordered in the way they’re loaded). Then, for example, you can trust the 3rd item in each collection represents the properties of the 3rd object you loaded.
Looping can be done with using either a Loop action or a For Each action.
- The Loop action simply loops until a condition is met that causes it to stop.
- The For Each action loops over the contents of a collection variable, but also has a condition property that will cause it to stop processing. When the loop is first entered, a snapshot of the collection is created and used for the duration of the iterations. As you’ll see in the implementation below, this disqualifies this action from being used against a stack type of collection variable, since any changes we make to the collection during a loop are not detected by the For Each action.
Implementation
Summary
The screenshot below is high-level example of how the workflow needs to be structured. Following is an overview of what’s going on.
- Push a baseline task to the stack to initialize it.
- Loop over the stack. A For Each action cannot be used against the stack collection since the collection is changed inside the loop (pushing and popping).
- Process the task. This includes popping the item(s) off of the stack and fetching the related tasks into a set of collections. As stated previously, a collection per column is required if you want to loop over a set of list items (or “records” or “objects”).
- Loop over the related tasks. This can be a For Each action, since the collections that hold related task data will not change during the loop. In this loop, finally I can do our date calculations and create the tasks. A search is done against each item to determine if related tasks exist. If so, you add it to the stack.
- Recalculate the size of the stack and repeat step 2 until the stack is empty.
Nintex Workflow Variables
| Variable | Type | Description |
|---|---|---|
| Stack | Collection | Stack for fetching related tasks. |
| Date Stack | Collection | Stack for passing the date as a parameter for calculations. |
| Baseline Id | Single line of text | Stores the ID of the starting task. The first value in the Stack collection. |
| Stack Count | Integer | Used by outer loop to determine to finish looping. Used by Collection operation action to count the Stack collection. Initialized to 1. |
| Related Id | Single line of text | Used to pop the top item off of the Stack. |
| Related Date | Date and Time | Used to pop the top item off of the Date Stack. |
| index | Number | Used for synchronizing the retrieval of column data across related task collections. |
| Id Collection | Collection | Used by query action to store the ID column for related tasks. |
| Title Collection | Collection | Used by query action to store the Title column for related tasks. |
| Days Collection | Collection | Used by query action to store the "Days from Related Task" column for related tasks. |
| Id | Single line of text | Used by Collection operation action to retrieve the ID column from the Id collection. |
| Title | Single line of text | Used by Collection operation action to retrieve the Title column from the Title collection. |
| Days | Number | Used by Collection operation action to retrieve the "Days from Related Task" column from the Days collection. |
| Date | Date and Time | Stores the result of the date calculation for a related task. |
| Related Count | Single line of text | Stores the result of a query to get the count of related tasks. This is text instead of a number to avoid unnecessary type casting since I'm using the OData feed and a Web request action. Therefore, if this variable has anything besides "0", the current ID and Date are pushed to the stacks. |
Sample list definition
| Column | Type | Description |
|---|---|---|
| Title | Single line of text | |
| Related Task | Lookup | Lookup column back to this list |
| Days From Related Task | Number |
Sample list data
| Title | Related Task | Days From Related Task |
|---|---|---|
| Gather documents for closing | Closing | -30 |
| Closing | Move In | -5 |
| Set up utilities | Move In | -30 |
| Move In |
Details
For further details, let’s dive in a little deeper into each of the steps summarized above.
- In order to “push” items into our stack, I’m simply adding items to the collection. This is achieved through the use of a Collection operation action using the Add option.
- Loop over the stack while the count of items in the stack collection is greater than 0. This is achieved using a variable, Stack Count, and evaluating against this in the Loop action.
- Process the task. In order to pop items off of the stack, you can again use the Collection operation action with the Pop option as shown in the following screenshot.
Once I have the Related Id, I can issue a query against our list to get all related tasks. This can be achieved using either a Query list action or a Query XML action against the OData list data feed. To store the related tasks, you’ll create a collection to hold each column that you’re concerned with. For my purposes, I’ll need at least the ID, Title and “Days From Related Task” columns in order to create my list items. This means I’ll have 3 collections (as shown above in the variables list).
- Loop over the related tasks. This loop action can be a For Each since I’ve already loaded the collections from the previous query (the collections are replaced by the query action in the previous step). This is where the index variable becomes important. The For Each action will iterate over a collection and store the index (incrementing number starting at 1) in a variable (named appropriately I might say).
The first actions in the loop fetch the values from the remaining collections based on this index:
Here’s an example of what the Get Title operation looks like:
Now that I have all the column values into variables, I can do my date calculation and create my task.
At this point I had a choice in how to implement my recursion algorithm. Either I could check for related tasks now (issue a count aggregate query) or later (let step 3 handle what to do if no related tasks were found from the query actions). I chose to issue a count query (a special quality and appropriate plug for the OData feed) to determine if the current item’s ID and date get pushed to the stack. This kept the stack cleaner in my opinion and simplified the retrieval logic in step 3. Pushing items to the stacks are accomplished in an identical manner as in step 1 using a Collection operation action with the Add option.
- Recalculate the size of the stack. This can be done using a Collection operation action with the Count option.
Final concerns
There are 2 final things that need to be mentioned here to set you up for success: Maintenance and Safe looping.
Maintenance
Implementing an algorithm like this in a workflow is an advanced technique and it’s complicated. Whenever I’m writing complicated C# code, I try my best to make it as simple and possible for the best long-term maintenance story. There are all sorts of software development techniques I can pull from to accomplish that, but I’ll oversimplify them here in a list and we can see how you can do the same thing in Nintex.
| Code | Nintex Workflow |
|---|---|
| Descriptive variable names | Descriptive variable names |
| Descriptive method names | Descriptive workflow action names |
| Refactor - Extract method | Descriptive-named action sets |
| Commenting | “User-generated” comments can be added to actions |
| Logging | “Log in history list” action |
| Unit testing | You’re kidding, right? No one has written the NintexUnit framework to my knowledge. If you want unit testing in a workflow, you’re most likely going to be using a full-blown Visual Studio workflow with .NET and not Nintex. |
Based on the table above, following is a list of things to make sure you do in a workflow to make it easier to maintain and support over time.
- In most cases, rename the workflow action to explain what it’s doing. Please don’t leave the default text there whenever you have more than 1 type of that action in a workflow. If you do a lot of logging in your workflows, rename each logging action to explain what’s inside. This will help you out later if you ever need to delete a variable that may be referenced in them.
- Use descriptive variable names. You can be even more verbose than you could be in code since you can have spaces in your variable names.
- You should also add your own comments to the actions whenever there is any doubt as to what a particular action is up to.
- Add logging actions whenever you think something may go wrong that might require research. A common thing to do would be to log your variables.
One way to do this is to add text and variables to the “Message to log on completion” text box on the action. This helps to reduce the number of logging actions in your workflow. I normally do this on actions that set variables, as in the screenshot below where I set the Stack Count variable.
- Use Action set actions to group related workflow actions whenever possible. You can then minimize them to make the workflow easier to navigate.
Safe looping
I can no longer avoid mentioning Safe looping, the elephant in the room. Safe looping is a Nintex global setting in Central Admin that makes sure someone doesn’t write an out of control looping algorithm (real easy to do that with the State Machine workflow action) that would bring a web front end server down with excessive load. It does this by inserting a pause of 5 minutes (derived from a timer job interval setting in your SharePoint environment) for each loop iteration during publish time. This means our Loop action will have pauses inserted while we’re “stacking it where they lacking it” (really sorry for that, but I couldn’t resist). This is not good, folks. When I was working with a client on a similar workflow, the difference between having safe looping turned off or on was whether the workflow would finish in 30 seconds or 30 minutes, respectively. Waiting 30 minutes for a workflow was not an acceptable option.
The solution is easier than you think, and you don’t have to sacrifice the safety of your farm in the process, either. All you need to do is make sure that safe looping is turned off when you publish your workflow. Since this setting requires an iisreset to take effect immediately, you could use the following steps to make sure you have a delay-free workflow at run-time.
- Turn Safe looping off
- Do an iisreset to force the setting to refresh
- Publish your workflow
- Turn Safe looping back on
- Do an iisreset to force the setting to refresh
This is obviously something you’d do outside of normal business hours, since users don’t tend to like the server going down (FAIL!). Since safe looping is enforced by modifying your workflow by injecting pauses during a publish, you can “safely” turn Safe looping back on and it won’t go back and mess with your workflow.
Is this setting a deal-breaker in your environment? Well, even though Nintex comes out of the box with Safe looping turned on, it is, after all, still a Central Admin setting in the UI, so this is definitely a supported configuration. As long as you exercise caution when overriding the default, it should be viewed like any other executable code you put in production. Hopefully you have multiple environments where you can do your workflow testing, so there should be a certain level of confidence by the time you’re ready to publish a workflow in a production farm, which is then followed by testing.
In my opinion Nintex workflows are supposed to help us solve a problem quickly, but I think this setting often gets directly in the way of that. Perhaps we’ll see some enhancements in the Nintex platform in the future to better support something like the steps I listed above, but it’s quite possible it won’t happen given the general direction of SharePoint 2013 moving away from web front ends being responsible for executing workflows.
Conclusion
It’s likely that your implementation of recursion will have nothing to do with a self-referencing lookup column, in which case the fashion in which you control how items get pushed and popped from your stacks would be completely different. Whatever the use case, I hope you see from my example how you could achieve recursion walking any tree of related items.
And, last but not least, I’ve exported a site workflow that you can use to inspect the design a little closer, or even use as a starting point for your recursion needs. I wrapped the whole workflow in an action set so you can save it as a snippet and move it to any other type (such as a reusable template or list workflow).
Cheers!
This is a great post. Thank you.
ReplyDelete