Embedding JSON Data into Yaml File

Embedding JSON Data into YAML file

I believe taking it into quotes should do the trick:

portslist: '[{"name":"ob1","port_type" ... }]'

Parse Json data and inserting into Yaml

I think your problem is this line:

yamlData['components']['component1']['data']['chef']['default'] = "$parsedJsonData"

The problem being the "$parsedJsonData" part.

This will call the toString() method of the interpolated data, which appears to be a Map.

To convert it to the JSON string representation, you could make use of the groovy.json.JsonOutput.html#toJson(java.util.Map) method in your pipeline instead.

If it is indeed a Map (or a few other types) it will be whitelisted by default by the Script Security Plugin (see here). If not, it may be blacklisted (see here).

import groovy.json.JsonOutput

// ...
yamlData['components']['component1']['data']['chef']['default'] = JsonOutput.toJson(parsedJsonData)

In a json embedded YAML file - replace only json values using Python

What you have is not YAML with embedded JSON, it is YAML with some the value for annotations being
in YAML flow style (which is a superset of JSON and thus closely resembles it).

This would be
YAML with embedded JSON:

api: v1
hostname: abc
metadata:
  name: test
  annotations: |
    {
      "ip" : "1.1.1.1",
      "login" : "fad-login",
      "vip" : "1.1.1.1",
      "interface" : "port1",
      "port" : "443"
    }

Here the value for annotations is a string that you can hand to a JSON parser.

You can just load the file, modify it and dump. This will change the layout
of the flow-style part, but that will not influence any following parsers:

import sys
import ruamel.yaml

file_in = Path('input.yaml')
    
yaml = ruamel.yaml.YAML()
yaml.preserve_quotes = True
yaml.width = 1024
data = yaml.load(file_in)
annotations = data['metadata']['annotations']
annotations['ip'] = type(annotations['ip'])('4.3.2.1')
annotations['vip'] = type(annotations['vip'])('1.2.3.4')
yaml.dump(data, sys.stdout)

which gives:

api: v1
hostname: abc
metadata:
  name: test
  annotations: {"ip": "4.3.2.1", "login": "fad-login", "vip": "1.2.3.4", "interface": "port1", "port": "443"}

The type(annotations['vip'])() establishes that the replacement string in the output has the same
quotes as the original.

ruamel.yaml currently doesn't preserve newlines in a flow style mapping/sequence.
If this has to go back into some repository with minimal chances, you can do:

import sys
import ruamel.yaml

file_in = Path('input.yaml')

def rewrite_closing_curly_brace(s):
    res = []
    for line in s.splitlines():
        if line and line[-1] == '}':
            res.append(line[:-1])
            idx = 0
            while line[idx] == ' ':
                idx += 1
            res.append(' ' * (idx - 2) + '}')
            continue
        res.append(line)
    return '\n'.join(res) + '\n'
    
yaml = ruamel.yaml.YAML()
yaml.preserve_quotes = True
yaml.width = 15
data = yaml.load(file_in)
annotations = data['metadata']['annotations']
annotations['ip'] = type(annotations['ip'])('4.3.2.1')
annotations['vip'] = type(annotations['vip'])('1.2.3.4')
yaml.dump(data, sys.stdout, transform=rewrite_closing_curly_brace)

which gives:

api: v1
hostname: abc
metadata:
  name: test
  annotations: {
    "ip": "4.3.2.1",
    "login": "fad-login",
    "vip": "1.2.3.4",
    "interface": "port1",
    "port": "443"
  }

Here the 15 for width is of course highly dependent on your file and might influence other lines if they
were longer. In that case you could leave that out, and make the wrapping
that rewrite_closing_curly_brace() does split and indent the whole flow style part.

Please note that your original, and the transformed output are, invalid YAML,
that is accepted by ruamel.yaml for backward compatibility. According to the YAML
specification the closing curly brace should be indented more than the start of annotation

How do I use json variables in a yaml file (Helm)

There's an example a bit like this in the docs for spring cloud dataflow but the format in their documentation has the quotes escaped.

I was able to recreate the error and get past it by changing the values file entry to:

service:
  spring_application_json:
    {
      "spring" : {
        "boot" : {
          "admin" : {
            "client" : {
              "enabled" : "false",
              "url" : "http://website1",
              "instance" : {
                "service-base-url" : "http://website2",
                "management-base-url" : "http://website3"
              }
            }
          }
        }
      }
    }

And the deployment entry to:

    - name: SPRING_APPLICATION_JSON
      value: {{ .Values.service.spring_application_json | toJson | quote }}

Notice no quotes around this part as that is handled anyway.

Put JSON data in yml file

To be technically correct, what you have shown there is neither YAML nor JSON but a Ruby Array containing Hashes.

In this syntax, it is not valid JSON. By fixing the syntactic issues (quoting the keys, removing trailing commas), you could get a mostly equivalent JSON representation (as long as you instruct the parser to read the Hash keys as Symbols rather than Strings).

If you chose the correct subset of your code, it could be valid YAML, although parsing it that way won't get the exact same data structure as with this YAML, the hash keys would be parsed as Strings rather than Symbols again.

Thus, to get an equivalent YAML representation of your data structure, you should just use YAML.dump to emit a YAML document from your existing data structure:

require 'yaml'
puts YAML.dump(BankDetails::Details)

How i can add a json entry inside a yaml file?

The easiest way in this case is to use a "literal block scalar".
This way you don't have to escape anything.

The literal block scalar is introduced with a | pipe sign.

http:
    port: "8081"
ABC:          
    CustomFieldJson: |
                      [
                      {
                     "FieldName": "uw_firm",
                     "FieldValue" :"NULL" 
                      },
                      {
                      "FieldName": "uw_type",
                      "FieldValue" :"Delegated"
                      }
                     ]

(More about quoting in YAML)

dump json into yaml

pyyaml.dump() has an allow_unicode option that defaults to None (all non-ASCII characters in the output are escaped). If allow_unicode=True, then it writes raw Unicode strings.

yaml.dump(data, ff, allow_unicode=True)

Bonus

You can dump JSON without encoding as follows:

json.dump(data, outfile, ensure_ascii=False)

Emit Python embedded object as native JSON in YAML document

Ok, so this was the solution I came up with. Generate the YAML with a placemarker ahead of time. The placemarker marks the place where the JSON should be inserted, and also defines the root-level indentation of the JSON block.

import os
import itertools
import json


def insert_json_in_yaml(pre_insert_yaml, key, obj_to_serialise):
    marker = '%s: null' % key
    marker_line = line_of_first_occurrence(pre_insert_yaml, marker)
    marker_indent = string_indent(marker_line)
    serialised = json.dumps(obj_to_serialise, indent=marker_indent + 4)
    key_with_json = '%s: %s' % (key, serialised)
    serialised_with_json = pre_insert_yaml.replace(marker, key_with_json)
    return serialised_with_json


def line_of_first_occurrence(basestring, substring):
    """
    return line number of first occurrence of substring
    """
    lineno = lineno_of_first_occurrence(basestring, substring)
    return basestring.split(os.linesep)[lineno]


def string_indent(s):
    """
    return indentation of a string (no of spaces before a nonspace)
    """
    spaces = ''.join(itertools.takewhile(lambda c: c == ' ', s))
    return len(spaces)


def lineno_of_first_occurrence(basestring, substring):
    """
    return line number of first occurrence of substring
    """
    return basestring[:basestring.index(substring)].count(os.linesep)


embedded_object = {
    "unitTypeCode": "",
    "unitNumber": "15",
    "levelTypeCode": "L",
    "roadNumber1": "810",
    "roadName": "HAY",
    "roadTypeCode": "ST",
    "localityName": "PERTH",
    "postcode": "6000",
    "stateTerritoryCode": "WA"
}
yaml_string = """
---

- properties:
    METHOD: GET
    TYPE: ADDRESS
    Request URL: /addresses
    testCaseId: TC2
  request: null
  after_request: another value
"""

>>> print(insert_json_in_yaml(yaml_string, 'request', embedded_object))
- properties:
    METHOD: GET
    TYPE: ADDRESS
    Request URL: /addresses
    testCaseId: TC2
  request: {
    "unitTypeCode": "",
    "unitNumber": "15",
    "levelTypeCode": "L",
    "roadNumber1": "810",
    "roadName": "HAY",
    "roadTypeCode": "ST",
    "localityName": "PERTH",
    "postcode": "6000",
    "stateTerritoryCode": "WA"
  }
  after_request: another value

What is the difference between YAML and JSON?

Technically YAML is a superset of JSON. This means that, in theory at least, a YAML parser can understand JSON, but not necessarily the other way around.

See the official specs, in the section entitled "YAML: Relation to JSON".

In general, there are certain things I like about YAML that are not available in JSON.

As @jdupont pointed out, YAML is visually easier to look at. In fact the YAML homepage is itself valid YAML, yet it is easy for a human to read.
YAML has the ability to reference other items within a YAML file using "anchors." Thus it can handle relational information as one might find in a MySQL database.
YAML is more robust about embedding other serialization formats such as JSON or XML within a YAML file.

In practice neither of these last two points will likely matter for things that you or I do, but in the long term, I think YAML will be a more robust and viable data serialization format.

Right now, AJAX and other web technologies tend to use JSON. YAML is currently being used more for offline data processes. For example, it is included by default in the C-based OpenCV computer vision package, whereas JSON is not.

You will find C libraries for both JSON and YAML. YAML's libraries tend to be newer, but I have had no trouble with them in the past. See for example Yaml-cpp.

Embedding JSON Data into Yaml File