Keeping Track of Your Threads

Since the release of Powershell v2 and the introduction of background tasks, multi threading has become an almost daily programming occurrence for me. If it's something you haven't delved into yet, take a look at the start-job commandlet and it's associated commandlets when you next fireup Powershell on your machine.

Now, multi threading is a fantastic way of getting things done in parallel quicker, when if they were running in serial it would take FOR EVER. For example, one script I changed recently went from taking 8 hours to complete (in serial) to 20 minutes in parallel. Obviously, it worked the server resources harder while it was doing it, but that's what they're there for right? However, before you start on your next multi threaded tool, think how much system resources each instance of your code is going to take up as the biggest problem I encountered is the dreaded OutOfMemory exception, and all of your threads fail. Not good.

So, to get around this, I suggest you pop the following function into any module or script you're going to use background tasks on. It allows to to specify how many concurrent tasks you want to be allowed to run at any one time, and also how long to wait before checking again. It's an iteravite function, so you just need to call it once from within the loop that you're spawning your threads from.

Function Threshold
{
	Param([int]$MaxConcurrent,[int]$PauseTime)
	$jobs = (get-job -state running | Measure-Object).count
 
	$RunningJobs = 0
 
	if($jobs -eq $null){
		$RunningJobs = 0
	}
	if($jobs -ne $null){
		$RunningJobs = $jobs
	}
 
	If ($RunningJobs -ge $MaxConcurrent){
		$TheMessage = "More than " + $MaxConcurrent.tostring() + " jobs running, waiting " + $PauseTime + " seconds" 	
		write-output $TheMessage 
		$Completed = (Get-Job -state Completed | Measure-Object).count
		if($Completed -ne $null){
			$TheMessage = "Currently there are " + $completed.tostring() + " completed jobs"
			write-output $TheMessage
		}
		$Running = get-job -state Running
		write-output "Current Running Jobs:"
		write-output $running
		start-sleep -seconds  $PauseTime
		Threshold $MaxConcurrent $PauseTime
	}
 
}

Just call the above code before you spawn a new thread, and it'll pause your script until there's space for the new threads to spawn. Happy multi threading!