Using Cloud-Init User Data

How to run cloud-init manually?

You can just run it like this:

/usr/bin/cloud-init -d init

This runs the cloud init setup with the initial modules. (The -d option is for debug) If want to run all the modules you have to run:

/usr/bin/cloud-init -d modules

Keep in mind that the second time you run these it doesn't do much since it has already run at boot time. To force to run after boot time you can run from the command line:

( cd /var/lib/cloud/ && sudo rm -rf * )

In older versions the equivalent of cloud-init init is:

/usr/bin/cloud-init start

You may also find this question useful although it applies to the older versions of cloud-init: How do I make cloud-init startup scripts run every time my EC2 instance boots?

The documentation for cloud init here just gives you examples. But it doesn't explain the command line options or each one of the modules, so you have to play around with different values in the config to get your desired results. Of course you can also look at the code.

Using cloud-init user data

It's an syntax error in your 3rd command:

- [ sh, -c, echo "=========hello world'=========" ]

This is a working user-data:

#cloud-config

runcmd:
- [ ls, -l, / ]
- [ sh, -xc, 'echo $(date) ": hello world!"' ]
- [ sh, -c, 'echo "=========hello world========="' ]
- [ touch, /home/ec2-user/hello.txt ]

final_message: "The system is finally up"

output : { all : '| tee -a /var/log/cloud-init-output.log' }

Notice that it shows cloud-init execution log only in /var/log/cloud-init.log. you should see output in /var/log/cloud-init-output.log after specifying the output directive.

How to validate a custom data file with cloud-init

I was wondering if anyone knew of a command to show if the file at the location above is valid syntactically?

To validate the config, check out the cloud-init schema subcommand. Something like this may be what you want:

On a live system:

cloud-init schema --system

To see the userdata passed, try:

cloud-init query userdata

To see everything passed, try:

cloud-init query --all

in AWS, is user_data executed before cloud-init?

Short answer:

A user_data value set to a shell script will cause the given shell script to be ran during in the final stage of cloud-init (and I believe after the cloud-init directives in the one-time folder you reference).

If you want to use a custom cloud-init directive and a shell script both in EC2 user_data property you need to use the multipart/mixed mime format https://aws.amazon.com/premiumsupport/knowledge-center/execute-user-data-ec2/

Long answer:

The user_data can hold raw data to be read through the EC2 meta data, a script or a cloud-init directive. Additionally, you can set it up as a multipart/mixed mime type and provide each of these.

If user_data is raw data, it can be fetched with a curl command inside the EC2 instance. It is up to the calling command to interpret the data, it can be whatever the user chooses.

[ec2-user ~]$ curl http://169.254.169.254/latest/user-data

If user_data is a script (e.g. #!/bin/bash in the first line), it is ran as a step in cloud-init in the final stage of cloud-init https://cloudinit.readthedocs.io/en/latest/topics/boot.html#final.

If user_data is a cloud-init directive (e.g. #cloud-config in the first line), it is ran as the specified cloud-init directive.

From https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html#user-data-cloud-init

"To pass cloud-init directives to an instance with user_data... enter your cloud-init directive text in the user_data text."

Like so

#cloud-config
repo_update: true
repo_upgrade: all

packages:
- httpd
- mariadb-server

runcmd:
- [ sh, -c, "amazon-linux-extras install -y lamp-mariadb10.2-php7.2 php7.2" ]
- systemctl start httpd
- sudo systemctl enable httpd
- [ sh, -c, "usermod -a -G apache ec2-user" ]
- [ sh, -c, "chown -R ec2-user:apache /var/www" ]
- chmod 2775 /var/www
- [ find, /var/www, -type, d, -exec, chmod, 2775, {}, \; ]
- [ find, /var/www, -type, f, -exec, chmod, 0664, {}, \; ]
- [ sh, -c, 'echo "<?php phpinfo(); ?>" > /var/www/html/phpinfo.php' ]

The multipart/mixed mime format is described here https://aws.amazon.com/premiumsupport/knowledge-center/execute-user-data-ec2/, with the example

Content-Type: multipart/mixed; boundary="//"
MIME-Version: 1.0

--//
Content-Type: text/cloud-config; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="cloud-config.txt"

#cloud-config
cloud_final_modules:
- [scripts-user, always]

--//
Content-Type: text/x-shellscript; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="userdata.txt"

#!/bin/bash
/bin/echo "Hello World" >> /tmp/testfile.txt
--//

user-data (cloud-init) script not executing on EC2

Cloud-init does not accept plain bash scripts, just like that. It's a beast that eats YAML file that defines your instance (packages, ssh keys and other stuff).

Using MIME you can also send arbitrary shell scripts, but you have to MIME-encode them.

$ cat my-boothook.txt
#!/bin/sh
echo "Hello World!"
echo "This will run as soon as possible in the boot sequence"

$ cat my-user-script.txt
#!/usr/bin/perl
print "This is a user script (rc.local)\n"

$ cat my-include.txt
# these urls will be read pulled in if they were part of user-data
# comments are allowed. The format is one url per line
http://www.ubuntu.com/robots.txt
http://www.w3schools.com/html/lastpage.htm

$ cat my-upstart-job.txt
description "a test upstart job"
start on stopped rc RUNLEVEL=[2345]
console output
task
script
echo "====BEGIN======="
echo "HELLO From an Upstart Job"
echo "=====END========"
end script

$ cat my-cloudconfig.txt
#cloud-config
ssh_import_id: [smoser]
apt_sources:
- source: "ppa:smoser/ppa"

$ ls
my-boothook.txt my-include.txt my-user-script.txt
my-cloudconfig.txt my-upstart-job.txt

$ write-mime-multipart --output=combined-userdata.txt \
my-boothook.txt:text/cloud-boothook \
my-include.txt:text/x-include-url \
my-upstart-job.txt:text/upstart-job \
my-user-script.txt:text/x-shellscript \
my-cloudconfig.txt

$ ls -l combined-userdata.txt
-rw-r--r-- 1 smoser smoser 1782 2010-07-01 16:08 combined-userdata.txt

The combined-userdata.txt is the file you want to paste there.

More info here:

https://help.ubuntu.com/community/CloudInit

Also note, this highly depends on the image you are using. But you say it is really cloud-init based image, so this applies. There are other cloud initiators which are not named cloud-init - then it could be different.

EC2 cloud-init script not updated on userdata update

The issue is that the runcmd script is copied to disk by the following directive (set by default):

cloud_config_modules:
- runcmd

But by default, cloud-init uses the instance repetition value. That means the runcmd module will be executed only once on an instance. To fix that, and to execute my script on every boot, the following configuration is required:

cloud_config_modules: 
- [runcmd, always] # copies the runcmd script from userdata to disk

cloud_final_modules:
- [scripts-user, always] # executes the runcmd script

Beware that I've omitted other modules here that may be required for the proper instance configuration. I just highlighted settings that resolved my issue.



Related Topics



Leave a reply



Submit