Extracting Variable in Yaml from a Shell Script

Extracting variable in YAML from a shell script

I'd recommend using a proper parser such as yq. Your task (last value of last element of acceleration_matrix) then becomes as simple as

yq eval '.acceleration_matrix[-1][-1]' nastran_acceleration_matrix.yml

How can I parse a YAML file from a Linux shell script?

My use case may or may not be quite the same as what this original post was asking, but it's definitely similar.

I need to pull in some YAML as bash variables. The YAML will never be more than one level deep.

YAML looks like so:

KEY:                value
ANOTHER_KEY:        another_value
OH_MY_SO_MANY_KEYS: yet_another_value
LAST_KEY:           last_value

Output like-a dis:

KEY="value"
ANOTHER_KEY="another_value"
OH_MY_SO_MANY_KEYS="yet_another_value"
LAST_KEY="last_value"

I achieved the output with this line:

sed -e 's/:[^:\/\/]/="/g;s/$/"/g;s/ *=/=/g' file.yaml > file.sh

• s/:[^:\/\/]/="/g finds : and replaces it with =", while ignoring :// (for URLs)
• s/$/"/g appends " to the end of each line
• s/ *=/=/g removes all spaces before =

Extract yaml file from a variable within a yaml file

I can't replicate your code exactly, but here's what I'm getting (OSX, bash v4.4.23, yq v2.1.0):

var=$(cat test.yaml | yq r - data.helm_values)
echo "$var"

with output

# replicas: number of pods
replicas: 1
# test
hello: world

If I forget the quotes, (echo $var), then:

# replicas: number of pods replicas: 1 # test hello: world

(no line breaks, no \n).

I can even further process it:

echo "$var" | yq r - hello
# => world

With yq v2.6.0 the command arguments have changed:

var=$(cat test.yaml | yq -r .data.helm_values)
echo "$var"

Linux shell get value of a field from a yml file

That is fairly easy, not using sed, but with appropriate shell tools. First, if you need to preserve sample in a variable for later use, then something like the following will work using bash substring replacement to isolate sample on the Test:/database: line:

$ db=$(grep -A3 'test:' database.yml | tail -n1); db=${db//*database: /}; echo "$db"
sample

or for a shorter solution that you can dump to the command line, remove the variable and command substitution and use a tool like cut:

$ grep -A3 'test:' database.yml | tail -n1 | cut -c 13-
sample

or, with awk, simply:

$ grep -A3 'test:' database.yml | tail -n1 | awk '{ print $2}'
sample

All of the different ways can be used inside command substitution (i.e. var=$(stuff) ) to store sample in var, it is just a matter of which you would rather use. I think you get the idea.

How to read a specific data from a yaml file inside a shell script

Here's an example with yq. All of the following assumes that the values do not contain newlines.

Given

$ cat test.yaml
---
test:
  config:
    abc: name1
    xyz: name2

then

yq e '.test.config | to_entries | map(.value) | .[]' test.yaml

outputs

name1
name2

---

You can read them into variables like

{ read -r var1; read -r var2; } < <(yq e '.test.config | to_entries | map(.value) | .[]' test.yaml)
declare -p var1 var2

declare -- var1="name1"
declare -- var2="name2"

---

I would read them into an associative array with the yaml key though:

declare -A conf
while IFS="=" read -r key value; do conf["$key"]=$value; done < <(
    yq e '.test.config | to_entries | map([.key, .value] | join("=")) | .[]' test.yaml
)
declare -p conf

declare -A conf=([abc]="name1" [xyz]="name2" )

Then you can write

echo "test config for abc is ${conf[abc]}"
# or
for var in "${!conf[@]}"; do printf "key %s, value %s\n" "$var" "${conf[$var]}"; done

---

I'm using "the Go implementation"

$ yq --version
yq (https://github.com/mikefarah/yq/) version 4.16.1

Get variables from a a file in bash

Consider using source, which will load the variables from that file

#!/bin/bash
source "$1"
echo "$database"
# mydb
echo "$table"
# mytable
echo "$partitons"
# p1,p2,p3

For more information about source see this superuser-question

Get value from yaml array in bash

This won't work in all systems, but

sed -n '/name: "kube-admin-local"/,/name:/s/.*client-certificate-data: \(.*\)/\1/p'

Should do it.

• -n: don't print unless explicitly stated
• /name: "kube-admin-local"/,/name:/: lines between those matches
• rest : substitute data and print

Extract from YAML values dynamically

YAML is not a programming language. It doesn't let you extract anything and it doesn't let you send anything, because it does not provide you with any kind of actions or processing instructions.

While you can reference nodes multiple times with anchors, that doesn't help you here because you would need to concatenate values, which is not something that is possible in YAML. A solution that would work looks like this (shortened for clarity):

map:
  &a FirstBin:
     source: value-field
     field-name: ServiceID
  &b SecondBin:
     source: value-field
     field-name: ServiceID
shouldDeleteOnNull: [*a, *b]

As you can see, I needed to make shouldDeleteOnNull a sequence rather than a scalar to make this work. This does not seem like much of an improvement.

Anything more sophisticated would need to be implemented in the code loading the file and therefore does not make sense to be discussed in a pure YAML context.

Extract each line using shell script and assign it to variables and save it individually as new file

This script is an example to read and parse the file (I called the data file "data.txt"):

#!/bin/sh

IFS=,

# read lines like: xxxxx,2017-06-26 13:12:53.750,-9.5949,124.6654,23.29,xxxx,yyyyy,mb,5.0,
while read xxx1 datetime lat long dep xxx2 xxx3 xxx4 mag; do

  # input lines are partly split
  echo "Read $datetime"
  echo "lat=$lat long=$long dep=$dep"

  # parse datetime field which is like 2017-06-26 13:12:53.750
  date=$(echo $datetime |cut -d" " -f1)  # -d tells the field separator
  time=$(echo $datetime |cut -d" " -f2)  # -f tells the field number to extract
  echo "date=$date time=$time"

  # extract year value from date, which is like 2017-06-26
  year=$(echo $date |cut -d"-" -f1)

  echo "year=$year"

  # go on this way to fill up all the variables...
  # ...left as an exercize...!

  # after this comment, down until the "done" keyword,...
  # ...you will have all the variables set, ready to be processed

done <data.txt

When this script is run, it shows the following:

user@machine:/tmp$ ./script.sh
Read 2017-06-26 13:12:53.750
lat=-9.5949 long=124.6654 dep=23.29
date=2017-06-26 time=13:12:53.750
year=2017
Read 2017-06-24 07:27:07.700
...
user@machine:/tmp$

As noted in some comment, read about the read command and cut(1) command. Hope it helps.

---

Related Topics