How to run cloud-init manually?
You can just run it like this:
/usr/bin/cloud-init -d init
This runs the cloud init setup with the initial modules. (The -d option is for debug) If want to run all the modules you have to run:
/usr/bin/cloud-init -d modules
Keep in mind that the second time you run these it doesn't do much since it has already run at boot time. To force to run after boot time you can run from the command line:
( cd /var/lib/cloud/ && sudo rm -rf * )
In older versions the equivalent of cloud-init init
is:
/usr/bin/cloud-init start
You may also find this question useful although it applies to the older versions of cloud-init: How do I make cloud-init startup scripts run every time my EC2 instance boots?
The documentation for cloud init here just gives you examples. But it doesn't explain the command line options or each one of the modules, so you have to play around with different values in the config to get your desired results. Of course you can also look at the code.
Using cloud-init user data
It's an syntax error in your 3rd command:
- [ sh, -c, echo "=========hello world'=========" ]
This is a working user-data:
#cloud-config
runcmd:
- [ ls, -l, / ]
- [ sh, -xc, 'echo $(date) ": hello world!"' ]
- [ sh, -c, 'echo "=========hello world========="' ]
- [ touch, /home/ec2-user/hello.txt ]
final_message: "The system is finally up"
output : { all : '| tee -a /var/log/cloud-init-output.log' }
Notice that it shows cloud-init execution log only in /var/log/cloud-init.log
. you should see output in /var/log/cloud-init-output.log
after specifying the output
directive.
How to validate a custom data file with cloud-init
I was wondering if anyone knew of a command to show if the file at the location above is valid syntactically?
To validate the config, check out the cloud-init schema subcommand. Something like this may be what you want:
On a live system:
cloud-init schema --system
To see the userdata passed, try:
cloud-init query userdata
To see everything passed, try:
cloud-init query --all
in AWS, is user_data executed before cloud-init?
Short answer:
A user_data
value set to a shell script will cause the given shell script to be ran during in the final stage of cloud-init (and I believe after the cloud-init directives in the one-time folder you reference).
If you want to use a custom cloud-init directive and a shell script both in EC2 user_data
property you need to use the multipart/mixed mime format https://aws.amazon.com/premiumsupport/knowledge-center/execute-user-data-ec2/
Long answer:
The user_data
can hold raw data to be read through the EC2 meta data, a script or a cloud-init directive. Additionally, you can set it up as a multipart/mixed mime type and provide each of these.
If user_data
is raw data, it can be fetched with a curl command inside the EC2 instance. It is up to the calling command to interpret the data, it can be whatever the user chooses.
[ec2-user ~]$ curl http://169.254.169.254/latest/user-data
If user_data
is a script (e.g. #!/bin/bash
in the first line), it is ran as a step in cloud-init in the final stage of cloud-init https://cloudinit.readthedocs.io/en/latest/topics/boot.html#final.
If user_data
is a cloud-init directive (e.g. #cloud-config
in the first line), it is ran as the specified cloud-init directive.
From https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html#user-data-cloud-init
"To pass cloud-init directives to an instance with user_data
... enter your cloud-init directive text in the user_data
text."
Like so
#cloud-config
repo_update: true
repo_upgrade: all
packages:
- httpd
- mariadb-server
runcmd:
- [ sh, -c, "amazon-linux-extras install -y lamp-mariadb10.2-php7.2 php7.2" ]
- systemctl start httpd
- sudo systemctl enable httpd
- [ sh, -c, "usermod -a -G apache ec2-user" ]
- [ sh, -c, "chown -R ec2-user:apache /var/www" ]
- chmod 2775 /var/www
- [ find, /var/www, -type, d, -exec, chmod, 2775, {}, \; ]
- [ find, /var/www, -type, f, -exec, chmod, 0664, {}, \; ]
- [ sh, -c, 'echo "<?php phpinfo(); ?>" > /var/www/html/phpinfo.php' ]
The multipart/mixed mime format is described here https://aws.amazon.com/premiumsupport/knowledge-center/execute-user-data-ec2/, with the example
Content-Type: multipart/mixed; boundary="//"
MIME-Version: 1.0
--//
Content-Type: text/cloud-config; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="cloud-config.txt"
#cloud-config
cloud_final_modules:
- [scripts-user, always]
--//
Content-Type: text/x-shellscript; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="userdata.txt"
#!/bin/bash
/bin/echo "Hello World" >> /tmp/testfile.txt
--//
user-data (cloud-init) script not executing on EC2
Cloud-init does not accept plain bash scripts, just like that. It's a beast that eats YAML file that defines your instance (packages, ssh keys and other stuff).
Using MIME you can also send arbitrary shell scripts, but you have to MIME-encode them.
$ cat my-boothook.txt
#!/bin/sh
echo "Hello World!"
echo "This will run as soon as possible in the boot sequence"
$ cat my-user-script.txt
#!/usr/bin/perl
print "This is a user script (rc.local)\n"
$ cat my-include.txt
# these urls will be read pulled in if they were part of user-data
# comments are allowed. The format is one url per line
http://www.ubuntu.com/robots.txt
http://www.w3schools.com/html/lastpage.htm
$ cat my-upstart-job.txt
description "a test upstart job"
start on stopped rc RUNLEVEL=[2345]
console output
task
script
echo "====BEGIN======="
echo "HELLO From an Upstart Job"
echo "=====END========"
end script
$ cat my-cloudconfig.txt
#cloud-config
ssh_import_id: [smoser]
apt_sources:
- source: "ppa:smoser/ppa"
$ ls
my-boothook.txt my-include.txt my-user-script.txt
my-cloudconfig.txt my-upstart-job.txt
$ write-mime-multipart --output=combined-userdata.txt \
my-boothook.txt:text/cloud-boothook \
my-include.txt:text/x-include-url \
my-upstart-job.txt:text/upstart-job \
my-user-script.txt:text/x-shellscript \
my-cloudconfig.txt
$ ls -l combined-userdata.txt
-rw-r--r-- 1 smoser smoser 1782 2010-07-01 16:08 combined-userdata.txt
The combined-userdata.txt is the file you want to paste there.
More info here:
https://help.ubuntu.com/community/CloudInit
Also note, this highly depends on the image you are using. But you say it is really cloud-init based image, so this applies. There are other cloud initiators which are not named cloud-init - then it could be different.
EC2 cloud-init script not updated on userdata update
The issue is that the runcmd script is copied to disk by the following directive (set by default):
cloud_config_modules:
- runcmd
But by default, cloud-init uses the instance
repetition value. That means the runcmd
module will be executed only once on an instance. To fix that, and to execute my script on every boot, the following configuration is required:
cloud_config_modules:
- [runcmd, always] # copies the runcmd script from userdata to disk
cloud_final_modules:
- [scripts-user, always] # executes the runcmd script
Beware that I've omitted other modules here that may be required for the proper instance configuration. I just highlighted settings that resolved my issue.
Related Topics
Sprof "Pltrel Not Found Error"
Making Pairs of Words Based on One Column
Is There Any Difference Between '=' and '==' Operators in Bash or Sh
Node.Js and Open Files Limit in Linux
Why Does The Stack Have to Be Page Aligned
Cannot Get Qwindow::Fromwinid to Work Properly
How to Automatically Start an Application That Needs X in Linux
How to Determine The Date-And-Time That a Linux Process Was Started
Monodevelop - Runs Only Using Sudo
Installing Zlib in Linux Server
Linux/Glibc. How to Use Fprintf in Signal Handler
Is There Any General Interfaces on Linux to Simulate Mouse Movements and Click
Get The Name of The Directory Where a Script Is Executed
Use Stdin from Within R Studio
Linux Slab Allocator and Cache Performance