Multi-threading Maven builds with PowerShell Jobs and a .NET Queue

I was recently given a copy of the script used on our Jenkins server to deploy our Java code base.  My boss wanted to get it working so devs could have a script to compile code on their local sandboxes, rather than wait for the next Jenkins run.  Honestly, I think I spent more time learning how Maven (the Java compiler) works than I did taking the script and changing all the paths around for local use.

So, the thing about working for a guy that knows more than you do is you constantly challenge  yourself to figure out how to make everything better; because you’re pretty sure he could do it better himself if he only had the time.  I knew exactly what he was going to say when I told him it took 45 minutes for the script to run through the list of projects and complete.  “Can you multithread that?”  I’d heard of PSJobs and that you could use them to spawn scripts, but hadn’t ever looked into them before.  So I said “I’ll get back to you on that” and starting researching.

I should say, my early attempts to do this were laughable.  I read up on PSJobs, took the code I’d already put together and wrapped it up in a ScriptBlock for Start-Job to spawn and ran it.  It worked so well it crashed my system!  Turns out my laptop didn’t want to run 40 instances of Maven all at the same time.  Oops.

Back to the drawing board and I found this Scripting Guy post: Avoid Overload by Scaling and Queuing PowerShell Background Jobs.  Genius!  Create a .NET Queue, stuff my paths into it and then pull one out on each run.  This was the first time I’d ever seen an event listener in a PowerShell script that wasn’t a GUI, pretty cool.  Everywhere else I’d read to just start a job, then use Wait-Job and Receive-Job.  This gave me asynchronicity, making it easier for me to check the log file for errors and retry the Maven task on the fly.

#Requires -Version 3.0
#v3 is needed to use PSJobs.

#Load the Forms assembly for use with Select-FolderDialog
[void][reflection.assembly]::Load("System.Windows.Forms, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089")

$Global:VerbosePreference = 'Continue'
$Global:MaxJobs = 3 #number of threads
$Global:MaxRetry = 3 #retry Maven on failure

#Function for Maven Installs required before running deploys
Function Global:BuildDefaultArtifactsFromQueue {  
    Write-Verbose ("Queue Count: {0}" -f $queue.count)
    $pomFile = $queue.Dequeue()
    #path to current working project
    $path=$pomFile.replace("\pom.xml","")
    $project = $path -match "\w+$"
    #get project folder name
    $project = $matches[0]
    
    #create job
    $job = Start-Job -Name $project -ScriptBlock { 
        Param($pomFile, $deployURL, $logPath, $path, $project) 
        $VerbosePreference = 'Continue'
        If(Test-Path $pomFile) {
            Set-Location $path
            Write-Verbose "Working on $project"
            #run Maven and pipe output to a log file
            & mvn clean | Out-File -FilePath "$logPath\install_$project.txt" -Append
            & mvn install | Out-File -FilePath "$logPath\install_$project.txt" -Append
            For ( $i = 0; $i -lt $MaxRetry; $i++ ) {
                #check for errors in the log and retry if found
                If (Select-String -Path "$logPath\install_$project.txt" -pattern "BUILD FAILURE") {
                    Remove-Item -Path "$logPath\install_$project.txt" -Force
                    Write-Verbose "ERROR WHILE INSTALLING $project - ATTEMPT $($i+1)!"
                    Set-Location $path
                    & mvn clean | Out-File -FilePath "$logPath\install_$project.txt" -Append
                    & mvn install | Out-File -FilePath "$logPath\install_$project.txt" -Append
                } Else { continue }
            }
            #warn if errors are still found
            If (Select-String -Path "$logPath\install_$project.txt" -pattern "BUILD FAILURE") {
                Write-Verbose "$project failed to install."
                Write-Verbose "Review - $logPath\install_$project.txt"
            }
        } Else { 
            Write-Verbose "BUILD FAILURE - $pomFile is missing"
        }
    } -ArgumentList $pomFile, $deployURL, $logPath, $path, $project

    #watch the job state and queue the next one on exit
    Register-ObjectEvent -InputObject $job -EventName StateChanged -Action { 
        $results = Receive-Job -Job $eventsubscriber.sourceobject 
        Write-Verbose "Removing event for $($eventsubscriber.sourceobject.Name)"            
        Remove-Job -Job $eventsubscriber.sourceobject 
        Unregister-Event $eventsubscriber.SourceIdentifier 
        Remove-Job -Name $eventsubscriber.SourceIdentifier 
        If ($queue.count -gt 0 -OR (Get-Job)) { 
            BuildDefaultArtifactsFromQueue 
        } ElseIf (!(Get-Job)) { 
            #at this point the installs should be done and deploys can be queued
            $End = New-Timespan $Start (Get-Date)                     
            Write-Verbose "$('Completed BuildDefaultArtifactsFromQueue in: {0}' -f $End)"
            $InputObject = $Global:BuildPOM
            ForEach($item in $InputObject) { 
                $item = "$sandboxURL\$item"
                Write-Verbose "Adding $item to queue"
                $queue.Enqueue($item) 
            } 
            For( $i = 0; $i -lt $Global:MaxJobs; $i++ ) { 
                BuildArtifactsFromQueue 
            }
        }
    } | Out-Null 
    Write-Verbose "Created Event for $project"
} 

Function Global:BuildArtifactsFromQueue {  
    Write-Verbose ("Queue Count: {0}" -f $queue.count)    
    $pomFile = $queue.Dequeue() 
    $path=$pomFile.replace("\pom.xml","")
    $project = $path -match "\w+$"
    $project = $matches[0]
    
    $job = Start-Job -Name $project -ScriptBlock { 
        Param($pomFile, $deployURL, $logPath, $path, $project) 
        $VerbosePreference = 'Continue'
        If(Test-Path $pomFile) {
            Set-Location $path
            Write-Verbose "Working on $project"
            & mvn clean | Out-File -FilePath "$logPath\deploy_$project.txt" -Append
            & mvn deploy "-Ddeployment.DeployURL=$deployURL" | Out-File -FilePath "$logPath\deploy_$project.txt" -Append
            For ( $i = 0; $i -lt $MaxRetry; $i++ ) {
                If (Select-String -Path "$logPath\deploy_$project.txt" -pattern "BUILD FAILURE") {
                    Remove-Item -Path "$logPath\deploy_$project.txt" -Force
                    Write-Verbose "ERROR WHILE DEPLOYING $project - ATTEMPT $($i+1)!"
                    Set-Location $path
                    & mvn clean | Out-File -FilePath "$logPath\deploy_$project.txt" -Append
                    & mvn deploy "-Ddeployment.DeployURL=$deployURL" | Out-File -FilePath "$logPath\deploy_$project.txt" -Append
                }
            }
            If (Select-String -Path "$logPath\deploy_$project.txt" -pattern "BUILD FAILURE") {
                Write-Verbose "$project failed to deploy."  
                Write-Verbose "Review -  $logPath\deploy_$project.txt"
            }
        } Else { 
            Write-Verbose "BUILD FAILURE - $pomFile is missing"
        }
    } -ArgumentList $pomFile, $deployURL, $logPath, $path, $project

    Register-ObjectEvent -InputObject $job -EventName StateChanged -Action { 
        $results = Receive-Job -Job $eventsubscriber.sourceobject 
        Write-Verbose "Removing Event for: $($eventsubscriber.sourceobject.Name)"            
        Remove-Job -Job $eventsubscriber.sourceobject 
        Unregister-Event $eventsubscriber.SourceIdentifier 
        Remove-Job -Name $eventsubscriber.SourceIdentifier 
        If ($queue.count -gt 0 -OR (Get-Job)) { 
            BuildArtifactsFromQueue 
        } ElseIf (!(Get-Job)) { 
            #TODO: Copy JAR files to JBoss
            $End = New-Timespan $Start (Get-Date)                     
            Write-Verbose "$('Completed BuildArtifactsFromQueue in: {0}' -f $End)"
            $ErrorLogs = Get-ChildItem -Path $logPath -recurse | Select-String -pattern "BUILD FAILURE" | group path | select name
            [void][System.Windows.Forms.MessageBox]::Show("$('Completed builds in: {0}' -f $End)`n`nSee error logs:`n$($ErrorLogs.Name)","Done")
        }            
    } | Out-Null 
    Write-Verbose "Created Event for $project" 
} 

Function Select-FolderDialog ($initialDirectory) {
    $OpenFolderDialog = New-Object System.Windows.Forms.FolderBrowserDialog -Property @{SelectedPath=$initialDirectory}
    $OpenFolderDialog.ShowDialog() | Out-Null
    return $OpenFolderDialog.SelectedPath
}

$Global:Start = Get-Date
$now = Get-Date -format "dd-MMM-yyyy HHmm" 
$Global:logPath = "C:\Development\Scripts\Maven\logs\$now"
#list of POM files for Maven Installs
$Global:DefaultArtifactBuildPOM = Get-Content "DefaultArtifactBuildPOM.txt"
#list of POM files for Maven Deploys
$Global:BuildPOM = Get-Content "ParentPOMValues.txt"
$Global:sandboxURL = Select-FolderDialog -initialDirectory "C:\Development\Sandbox"
$Global:deployURL = "C:\Development\Deploy"

mkdir $logPath -f | Out-Null 
#delete old deploys if they exist
If (!(Test-Path $Global:deployURL)) {
    mkdir $Global:deployURL -f | Out-Null
} Else { Remove-Item $Global:deployURL\* -force -recurse }
If (Test-Path C:\Users\$env:username\.m2) {
    Get-ChildItem -Path C:\Users\$env:username\.m2 | Remove-Item -force -recurse
}

$Global:queue = [System.Collections.Queue]::Synchronized( (New-Object System.Collections.Queue) ) 

$InputObject = $Global:DefaultArtifactBuildPOM
#queue up the first list
ForEach($item in $InputObject) {
    $item = "$sandboxURL\$item"
    Write-Verbose "Adding $item to queue"
    $queue.Enqueue($item) 
} 

#spawn the first set of jobs
For( $i = 0; $i -lt $Global:MaxJobs; $i++ ) { 
    BuildDefaultArtifactsFromQueue
}
Advertisements
This entry was posted in DevOps, Programming and tagged , , , , , . Bookmark the permalink.

2 Responses to Multi-threading Maven builds with PowerShell Jobs and a .NET Queue

  1. Pingback: Multi-threading Maven builds with PowerShell Jobs and a .NET Queue | Dinesh Ram Kali.

  2. Pingback: Multi-threading Maven builds with PowerShell Jobs and a .NET Queue | Maven Advanced

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s