Conditionally apply pipeline step depending on external value
How about this approach:
mtcars %>%
tibble::rownames_to_column(var = "model") %>%
filter(if(applyfilter== 1) grepl(x = model, pattern = "Merc") else TRUE) %>%
group_by(am) %>%
summarise(meanMPG = mean(mpg))
This means grepl is only evaluated if the applyfilter is 1, otherwise the filter simply recycles a TRUE.
Or another option is to use {}:
mtcars %>%
tibble::rownames_to_column(var = "model") %>%
{if(applyfilter == 1) filter(., grepl(x = model, pattern = "Merc")) else .} %>%
group_by(am) %>%
summarise(meanMPG = mean(mpg))
There's obviously another possible approach in which you would simply break the pipe, conditionally do the filter and then continue the pipe (I know OP didn't ask for this, just want to give another example for other readers)
mtcars %<>%
tibble::rownames_to_column(var = "model")
if(applyfilter == 1) mtcars %<>% filter(grepl(x = model, pattern = "Merc"))
mtcars %>%
group_by(am) %>%
summarise(meanMPG = mean(mpg))
Conditional step in pipeline executed only based on output of a previous setp
you can declare the gitStatus outside the pipeline block as below
def gitStatus
pipeline {
stages {
stage('Definitions updated') {
steps {
sh "./gradlew updateDefinitions"
gitStatus = sh(returnStdout: true, script: 'git status').trim()
## how to expose gitStatus to the outside
}
}
stage ('Create PR') {
when {
// Only say hello if a "status returned something"
## how to use the gitStatus to check against a certain output
expression { gitStatus == 'SOMETHING'' }
}
steps {
sh "git add ."
etc...
}
}
}
}
Conditional step/stage in Jenkins pipeline
Doing the same in declarative pipeline syntax, below are few examples:
stage('master-branch-stuff') {
when {
branch 'master'
}
steps {
echo 'run this stage - ony if the branch = master branch'
}
}
stage('feature-branch-stuff') {
when {
branch 'feature/*'
}
steps {
echo 'run this stage - only if the branch name started with feature/'
}
}
stage('expression-branch') {
when {
expression {
return env.BRANCH_NAME != 'master';
}
}
steps {
echo 'run this stage - when branch is not equal to master'
}
}
stage('env-specific-stuff') {
when {
environment name: 'NAME', value: 'this'
}
steps {
echo 'run this stage - only if the env name and value matches'
}
}
More effective ways coming up -
https://issues.jenkins-ci.org/browse/JENKINS-41187
Also look at -
https://jenkins.io/doc/book/pipeline/syntax/#when
The directive beforeAgent true can be set to avoid spinning up an agent to run the conditional, if the conditional doesn't require git state to decide whether to run:
when { beforeAgent true; expression { return isStageConfigured(config) } }
Release post and docs
UPDATE
New WHEN Clause
REF: https://jenkins.io/blog/2018/04/09/whats-in-declarative
equals - Compares two values - strings, variables, numbers, booleans -
and returns true if they’re equal. I’m honestly not sure how we missed
adding this earlier! You can do "not equals" comparisons using the not
{ equals ... } combination too.changeRequest - In its simplest form, this will return true if this
Pipeline is building a change request, such as a GitHub pull request.
You can also do more detailed checks against the change request,
allowing you to ask "is this a change request against the master
branch?" and much more.buildingTag - A simple condition that just checks if the Pipeline is
running against a tag in SCM, rather than a branch or a specific
commit reference.tag - A more detailed equivalent of buildingTag, allowing you to check
against the tag name itself.
Conditional input step in declarative pipeline
I think you are using the wrong syntax here. The input { … } is only valid as a directive (outside of the steps directly below stage). What you want to use is the input step which is described here. Basically you just need to remove the curly braces and put it into a script within steps:
stage("stage") {
steps {
script {
if (params.apply_plan != 'yes') {
input message: 'Apply this plan?'
}
}
withAWS(region: 'us-east-1', role: assume_role) {
dir(path: tf_dir) {
sh "make apply"
}
}
}
}
Is it possible to pipe conditionally in Powershell, i.e. execute an element of a pipeline only if a condition is met?
Sorry, I didn't mean to abandon this question. The answers that were posted weren't what I was driving at, but I figured out a way to do it shortly after posting, and didn't come back to the site for a long time. Since a solution hasn't been posted, here's what I came up with. It's not quite what I had in mind when I asked the question and it isn't too pretty, but apparently it's the only way to do it:
<statement> | <filter1> | foreach {if (<condition>) {$_ | <filter2>} else {$_} | <filter3> | <filter4> | <filter5>
So, in the example, the line
|where {$_.psiscontainer} `
would be changed to
|foreach {if (-not $files) {$_ | where {$_.psiscontainer}} else {$_}} `
and
|where {$_.isinherited -eq 'False'} `
would be changed to
|foreach {if (-not $inherited) {$_ | where {$_.isinherited -eq 'False'}} else {$_}} `
(Yes, normally I'd write that as |foreach {if ($files) {$_} else {$_ | where {$_.psiscontainer}}}, and |foreach {if ($inherited) {$_} else {$_ | where {$_.isinherited -eq 'False'}}} but I did it this way for clarity.)
I was hoping there might be something more elegant, that would evaluate a condition in front of the filter once to determine whether to execute or skip a stage of the pipeline. Something like this:
<statement> | <filter1> | if (<condition>) {<filter2>} | <filter3>
(a special case of if, not the usual meaning; a different keyword could be used), or maybe
<statement> | <filter1> | (<condition>) ? <filter2> | <filter3>
$_ would be invalid in the condition, unless it's defined outside the current pipeline, for example if the pipeline is contained within a switch statement, $_ in the <condition> would refer the switch statement's $_.
I think I'll make a feature suggestion to Microsoft. This would not only make the code more elegant, it would be more efficient as well, because if it's a built-in feature, <condition> could be evaluated once for the entire pipeline, rather then testing the same independent condition in each iteration.
Jenkins Pipeline with conditional When expression of choice parameters
From the docs:
Any parameters provided as part of the input submission will be available in the environment for the rest of the stage.
This means that your parameter CHOICE does not exist in the other stages. If you want to have a parameter that's available on all the stages, you can define a parameter outside of the stage, i.e.:
pipeline {
agent any
parameters {
choice(choices:['Hello','Bye'], description: 'Users Choice', name: 'CHOICE')
}
stages {
stage('Init') {
steps('Log-in') {
echo 'Log-in'
}
}
stage('Manual Step') {
steps('Input') {
echo "choice: ${CHOICE}"
echo "choice params.: " + params.CHOICE
echo "choice env: " + env.CHOICE
}
}
stage('Hello') {
when {
expression { env.CHOICE == 'Hello' }
}
steps('Execute') {
echo 'Say Hello'
}
}
stage('Bye') {
when {
expression {env.CHOICE == 'Bye'}
}
steps('Execute'){
echo 'Say Bye'
}
}
}
}
This will behave as expected. The difference is that the job won't ask you for input, instead, you will provide the wanted parameters before pressing build.
Conditionally Include Aggregation Pipeline Stages
What you mean to do is build the whole pipeline depending on the options provided. It' is just a data structure after-all.
You are also incorrectly testing for an "array" and you should be using instanceof because typeof would actually return "object" and not "array".
Moreover you really want that condition in the first pipeline stage to optimally select the documents as well, in addition to being added after $unwind where needed:
var pipeline = [
{ $match:
Object.assign(
{ 'shop.nameSlug' : req.query.nameSlug },
(req.query.status)
? { "status.status": (req.query.status instanceof Array)
? { "$in": req.query.status } : req.query.status }
: {}
)
},
{ $unwind: "$status" },
...(
(req.query.status)
? [{ "$match": {
"status.status": (req.query.status instanceof Array)
? { "$in": req.query.status } : req.query.status
}}]
: []
),
{ $group: {
_id: "$_id",
status: { $addToSet: "$status" },
number: { $first: "$number" },
date: { $first: "$date" },
comment: { $first: "$comment" }
}}
];
Order.aggregate(pipeline).exec(function(err, orders){
})
Given a req object with something present in status you get:
// Example stucture
var req = {
query: {
nameSlug: "Bill",
status: "A"
},
};
// Pipeline output as:
[
{
"$match" : {
"shop.nameSlug" : "Bill",
"status.status" : "A"
}
},
{
"$unwind" : "$status"
},
{
"$match" : {
"status.status" : "A"
}
},
{
"$group" : {
"_id" : "$_id",
"status" : {
"$addToSet" : "$status"
},
"number" : {
"$first" : "$number"
},
"date" : {
"$first" : "$date"
},
"comment" : {
"$first" : "$comment"
}
}
}
]
With an array:
var req = {
query: {
nameSlug: "Bill",
status: ["A","B"]
},
};
// Pipeline output as:
[
{
"$match" : {
"shop.nameSlug" : "Bill",
"status.status" : {
"$in" : [
"A",
"B"
]
}
}
},
{
"$unwind" : "$status"
},
{
"$match" : {
"status.status" : {
"$in" : [
"A",
"B"
]
}
}
},
{
"$group" : {
"_id" : "$_id",
"status" : {
"$addToSet" : "$status"
},
"number" : {
"$first" : "$number"
},
"date" : {
"$first" : "$date"
},
"comment" : {
"$first" : "$comment"
}
}
}
]
And with nothing:
var req = {
query: {
nameSlug: "Bill",
//status: ["A","B"]
},
};
// Pipeline output as:
[
{
"$match" : {
"shop.nameSlug" : "Bill"
}
},
{
"$unwind" : "$status"
},
{
"$group" : {
"_id" : "$_id",
"status" : {
"$addToSet" : "$status"
},
"number" : {
"$first" : "$number"
},
"date" : {
"$first" : "$date"
},
"comment" : {
"$first" : "$comment"
}
}
}
]
So you can see where the parts are conditionally included depending on the values provided.
Using $filter
You really should be using $filter here instead. It's a lot more efficient than $unwind and you really are not grouping anything. All you really want is filtered arrays. So that's all you conditionally add:
var pipeline = [
{ $match:
Object.assign(
{ 'shop.nameSlug' : req.query.nameSlug },
(req.query.status)
? { "status.status": (req.query.status instanceof Array)
? { "$in": req.query.status } : req.query.status }
: {}
)
},
...(
(req.query.status)
? [{ "$addFields": {
"status": {
"$filter": {
"input": "$status",
"cond": {
[(req.query.status instanceof Array) ? "$in" : "$eq"]:
[ "$$this.status", req.query.status ]
}
}
}
}}]
: []
)
];
The choice there is between $in and $eq for the comparison test, depending on what is supplied. You can optionally wrap the whole thing in $setUnion if you "really mean" to use a "set" in the result. But it's generally looking like you simply want to "filter" values out of the array.
With the same expectations of a single value:
var req = {
query: {
nameSlug: "Bill",
//status: ["A","B"]
status: "A"
},
};
/* 1 */
[
{
"$match" : {
"shop.nameSlug" : "Bill",
"status.status" : "A"
}
},
{
"$addFields" : {
"status" : {
"$filter" : {
"input" : "$status",
"cond" : {
"$eq" : [
"$$this.status",
"A"
]
}
}
}
}
}
]
An array:
var req = {
query: {
nameSlug: "Bill",
status: ["A","B"]
},
};
/* 1 */
[
{
"$match" : {
"shop.nameSlug" : "Bill",
"status.status" : {
"$in" : [
"A",
"B"
]
}
}
},
{
"$addFields" : {
"status" : {
"$filter" : {
"input" : "$status",
"cond" : {
"$in" : [
"$$this.status",
[
"A",
"B"
]
]
}
}
}
}
}
]
Or nothing:
var req = {
query: {
nameSlug: "Bill",
//status: ["A","B"]
},
};
/* 1 */
[
{
"$match" : {
"shop.nameSlug" : "Bill"
}
}
]
Where if you don't need to filter, then you simply drop the additional pipeline stage.
How can I conditionally use a container on Azure Pipelines?
You can't do this with variables from matrix, since template expressions (${{ }}) are rendered very early and they don't have access to those variables. See this page to get an idea of how pipeline is processed: Pipeline run sequence
1. First, expand templates and evaluate template expressions.
...
4. For each job selected to run, expand multi-configs (strategy: matrix or strategy: parallel in YAML) into multiple runtime jobs.
But you can achieve desired result by using runtime parameters, which are available to the template expressions:
parameters:
- name: matrix
type: object
default:
GCC_10:
CXX: g++-10
VM_IMAGE: ubuntu-20.04
GCC_9:
CXX: g++-9
VM_IMAGE: ubuntu-20.04
Clang_3_7:
CXX: clang++-3.7
VM_IMAGE: ubuntu-20.04
CONTAINER: ubuntu:16.04
stages:
- stage: Test
jobs:
- ${{ each item in parameters.matrix }}:
- job: Linux_${{ item.Key }}
pool:
vmImage: ${{ item.Value.VM_IMAGE }}
${{ if item.Value.CONTAINER }}:
container:
image: ${{ item.Value.CONTAINER }}
options: --name ci-container -v /usr/bin/docker:/tmp/docker:ro
steps:
- bash: echo CXX = ${{ item.Value.CXX }}
This will be rendered to
parameters:
- name: matrix
type: object
default:
GCC_10:
CXX: g++-10
VM_IMAGE: ubuntu-20.04
GCC_9:
CXX: g++-9
VM_IMAGE: ubuntu-20.04
Clang_3_7:
CXX: clang++-3.7
VM_IMAGE: ubuntu-20.04
CONTAINER: ubuntu:16.04
stages:
- stage: Test
jobs:
- job: Linux_GCC_10
pool:
vmImage: ubuntu-20.04
steps:
- task: Bash@3
inputs:
targetType: inline
script: echo CXX = g++-10
- job: Linux_GCC_9
pool:
vmImage: ubuntu-20.04
steps:
- task: Bash@3
inputs:
targetType: inline
script: echo CXX = g++-9
- job: Linux_Clang_3_7
pool:
vmImage: ubuntu-20.04
container:
image: ubuntu:16.04
options: --name ci-container -v /usr/bin/docker:/tmp/docker:ro
steps:
- task: Bash@3
inputs:
targetType: inline
script: echo CXX = clang++-3.7
How to ignore a mongo pipeline stage conditionally based on field values?
Query
- map over items and add the product(
$mergeObjects) in all members - you dont need condition, if array is empty, map will do nothing, leaving the array empty.
*in general you cant skip a $addFields stage but you can keep the old value based on condition, here if array is empty $map does nothing anyway, so we dont even need condition to keep the old value.
Test code here
db.collection.aggregate([
{
"$set": {
"items": {
"$map": {
"input": "$items",
"in": {
"$mergeObjects": [
"$$i",
{
"product": {
"title": "some title",
"code": "some code"
}
}
]
},
"as": "i"
}
}
}
}
])
Related Topics
How to Manipulate Null Elements in a Nested List
Getting Both Column Counts and Proportions in the Same Table in R
Shading Confidence Intervals Manually with Ggplot2
Trycatch with Parlapply (Parallel Package) in R
Warning: Unable to Access Index for Repository Https://Www.Stats.Ox.Ac.Uk/Pub/Rwin/Src/Contrib:
Convert a Vector into a List, Each Element in the Vector as an Element in the List
Provide Shades Between Dates on X Axis
Remove White Space Between Plots and Table in Grid.Arrange
How to Plot the Linear Regression in R
Controlling Both the Major and Minor Grid Lines on the Y Axis
Adding Regression Line Equation and R2 on Separate Lines Graph
Check If Character String Is a Valid Color Representation
Cumulative Count of Unique Values in R
How to Reorder Factor Levels in a Tidy Way
Grid.Arrange Using List of Plots