Use placeholders in yaml
Context
- YAML version 1.2
- user wishes to
- include variable placeholders in YAML
- have placeholders replaced with computed values, upon
yaml.load
- be able to use placeholders for both YAML mapping keys and values
Problem
- YAML does not natively support variable placeholders.
- Anchors and Aliases almost provide the desired functionality, but these do not work as variable placeholders that can be inserted into arbitrary regions throughout the YAML text. They must be placed as separate YAML nodes.
- There are some add-on libraries that support arbitrary variable placeholders, but they are not part of the native YAML specification.
Example
Consider the following example YAML. It is well-formed YAML syntax, however it uses (non-standard) curly-brace placeholders with embedded expressions.
The embedded expressions do not produce the desired result in YAML, because they are not part of the native YAML specification. Nevertheless, they are used in this example only to help illustrate what is available with standard YAML and what is not.
part01_customer_info:
cust_fname: "Homer"
cust_lname: "Himpson"
cust_motto: "I love donuts!"
cust_email: homer@himpson.org
part01_government_info:
govt_sales_taxrate: 1.15
part01_purchase_info:
prch_unit_label: "Bacon-Wrapped Fancy Glazed Donut"
prch_unit_price: 3.00
prch_unit_quant: 7
prch_product_cost: "{{prch_unit_price * prch_unit_quant}}"
prch_total_cost: "{{prch_product_cost * govt_sales_taxrate}}"
part02_shipping_info:
cust_fname: "{{cust_fname}}"
cust_lname: "{{cust_lname}}"
ship_city: Houston
ship_state: Hexas
part03_email_info:
cust_email: "{{cust_email}}"
mail_subject: Thanks for your DoughNutz order!
mail_notes: |
We want the mail_greeting to have all the expected values
with filled-in placeholders (and not curly-braces).
mail_greeting: |
Greetings {{cust_fname}} {{cust_lname}}!
We love your motto "{{cust_motto}}" and we agree with you!
Your total purchase price is {{prch_total_cost}}
Explanation
Below is an inline image that illustrates the example with colored regions in green, yellow and red.
The substitutions marked in GREEN are readily available in standard YAML, using anchors, aliases, and merge keys.
The substitutions marked in YELLOW are technically available in standard YAML, but not without a custom type declaration, or some other binding mechanism.
The substitutions marked in RED are not available in standard YAML. Yet there are workarounds and alternatives; such as through string formatting or string template engines (such as python's
str.format
).
Details
Templates with variable placeholders is a frequently-requested YAML feature.
Routinely, developers want to cross-reference content in the same YAML file or transcluded YAML file(s).
YAML supports anchors and aliases, but this feature does not support arbitrary placement of placeholders and expressions anywhere in the YAML text. They only work with YAML nodes.
YAML also supports custom type declarations, however these are less common, and there are security implications if you accept YAML content from potentially untrusted sources.
YAML addon libraries
There are YAML extension libraries, but these are not part of the native YAML spec.
- Ansible
- https://docs.ansible.com/ansible-container/container_yml/template.html
- (supports many extensions to YAML, however it is an Orchestration tool, which is overkill if you just want YAML)
- https://github.com/kblomqvist/yasha
- https://bitbucket.org/djarvis/yamlp
Workarounds
- Use YAML in conjunction with a template system, such as Jinja2 or Twig
- Use a YAML extension library
- Use
sprintf
orstr.format
style functionality from the hosting language
Alternatives
- YTT YAML Templating essentially a fork of YAML with additional features that may be closer to the goal specified in the OP.
- Jsonnet shares some similarity with YAML, but with additional features that may be closer to the goal specified in the OP.
See also
Here at SO
- YAML variables in config files
- Load YAML nested with Jinja2 in Python
- String interpolation in YAML
- how to reference a YAML "setting" from elsewhere in the same YAML file?
- Use YAML with variables
- How can I include a YAML file inside another?
- Passing variables inside rails internationalization yml file
- Can one YAML object refer to another?
- is there a way to reference a constant in a yaml with rails?
- YAML with nested Jinja
- YAML merge keys
- YAML merge keys
Outside SO
- https://learnxinyminutes.com/docs/yaml/
- https://github.com/dreftymac/awesome-yaml#variables
- https://duckduckgo.com/?q=yaml+variables+in+config+file&t=h_&ia=web
How do I read a yaml file and outputting the yaml file values as environment variables?
If you have Python installed in your environment and can install ruamel.yaml
in there you can source the output of the following one-liner:
python -c 'from pathlib import Path; from ruamel.yaml import YAML; print("".join([f"{k}={v}\n" for k, v in YAML().load(Path("example.yaml"))["env_variables"].items()]))'
Its output is:
DATABASE_CONNECTION_ADDRESS=localhost
DATABASE_PORT=5432
DATABASE_NAME=a-db
DATABASE_USERNAME=user
DATABASE_PASSWORD=password
IS_DEBUG=false
GS_BUCKET_NAME=image-bucket
As Jeff Schaller suggested you probably want to quote the values and escape any single quotes that might occur in the string. This can easily be achieved by changing {v}
into {v!r}
in the one-liner.
As program:
#!/usr/bin/env python3
from pathlib import Path
from ruamel.yaml import YAML
file_in = Path("example.yaml")
yaml = YAML()
env_data = yaml.load(file_in)["env_variables"]
print("".join([f"{k}={v!r}\n" for k, v in env_data.items()]))
How do I use a variable from an application.yml file in my normal code?
You can use this:
@Value("${your.path.yml.string}")
private String x;
YML:
your:
path:
yml:
string: hello
x will be "hello"
Declare and use variable in YAML (Percy.io)
I wanted to toss this suggestion in, you it looks like you're doing this for the base URL of the site you're snapshotting. You can pass a --base-url
flag to the snapshot
command:
$ npx percy snapshot --base-url http://localhost:8080 snapshots.yml
snapshots.yml
could be:
- name: Name 1
url: /page1
- name: Name 2
url: /page2
- name: Name 3
url: /page3
## or
- url: /page1
- url: /page2
- url: /page3
Relevant Percy CLI PR
How do I use variables in a YAML file?
You could use ERB.
For example:
template = ERB.new File.new("path/to/config.yml.erb").read
processed = YAML.load template.result(binding)
You can read more on binding here: ruby metaprogramming.
In Azure yaml, how to use local variables, conditional variables and template variables together?
How can I combine these 3?
In Pipeline YAML, variables can only be defined once at a stage or root level.
To meet your requirements, you need to define three type of variables in the same variables field.
Here is an example:
parameters:
- name: environment
displayName: Test
type: string
values:
- dev
- test
- prod
variables:
- name: buildPlatform
value: 'Any CPU'
- name: buildConfiguration
value: 'Release'
- template: variables.yml
- ${{ if eq(parameters.environment, 'dev') }}:
- name: environment
value: development
- ${{ if eq(parameters.environment, 'test') }}:
- name: environment
value: test
- ${{ if eq(parameters.environment, 'prod') }}:
- name: environment
value: prod
Related Topics
Is There a Pry Debug Setup That Works with Ruby 2.0
How to Express 'Infinite Time'
No Implicit Conversion from Nil to Integer - When Trying to Add Anything to Array
Nil.To_JSON Cannot Be Parsed Back to Nil
Which Ruby Classes Support .Clone
Find Where Associated Records Exist
How to Dynamically Define a Method as Private
How to Add a Custom Log Level to Logger in Ruby
Can Someone Please Explain Class << Self to Me
How to Remove Blank Values Params from Query String
Is Subclassing a User Model Really Bad to Do in Rails
Using S3 Presigned-Url for Upload a File That Will Then Have Public-Read Access
Rails Generate Error: No Such File or Directory - Getcwd
What's the Best Way to Implement Acls to a Rails Application