How to Find Out Why My Storage Space on Amazon Ec2 Is Full

How can I find out why my storage space on Amazon EC2 is full?

Well, I think its one (or more) logfiles which have grown too large and need to be removed/backupped. I would suggest going after the big files first. So find all files greater than 10 MB (10 MB is a big enough file size, you can choose +1M for 1MB similarly)

sudo find / -type f -size +10M -exec ls -lh {} \;

and now you can identify which ones are causing the trouble and deal with them accordingly.

As for your original du -a / | sort -n -r | head -n 10 command, that won't work since it is sorting by size, and so, all ancestor directories of a large file will go up the pyramid, while the individual file will most probably be missed.

Note: It should be pretty simple to notice the occurence of similar other log files/binaries in the location of the files you so find, so as a suggestion, do cd in to the directory containing the original file to cleanup more files of the same kind. You can also iterate with the command for files with sizes greater than 1MB next, and so on.

Running out of disk space in Amazon EC2, can't find what I am using my storage for

/dev/xvda1 is your root volume. The AMI you listed has a default root volume size of 20GB as you can see here:

Describe the image and get it's block device mappings:

aws ec2 describe-images --image-ids ami-3b0c205e --region us-east-2 | jq .Images[].BlockDeviceMappings[]

Look at the volume size

{
  "DeviceName": "/dev/sda1",
  "Ebs": {
    "Encrypted": false,
    "DeleteOnTermination": true,
    "VolumeType": "gp2",
    "VolumeSize": 20,
    "SnapshotId": "snap-03341b1ff8ee47eaa"
  }
}
{
  "DeviceName": "/dev/sdb",
  "VirtualName": "ephemeral0"
}
{
  "DeviceName": "/dev/sdc",
  "VirtualName": "ephemeral1"
}

When launched with the correct volume size of 20GB there is plenty of free space (10GB)

root@ip-10-100-0-64:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            488M     0  488M   0% /dev
tmpfs           100M  3.1M   97M   4% /run
/dev/xvda1       20G  9.3G   11G  49% /
tmpfs           496M     0  496M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           496M     0  496M   0% /sys/fs/cgroup
tmpfs           100M     0  100M   0% /run/user/1000

It appears the issue here is the instance was launched with 10GB (somehow, I didn't think this was possible) of storage instead of the default 20GB.

Running out of disk space EC2

Amazon Elastic Block Storage (EBS) is a service that provides virtual disks for use with Amazon EC2. It is network-attached storage that persists even when an EC2 instance is stopped or terminated.

When launching an Amazon EC2 instance, a boot volume is automatically attached to the instance. The contents of the boot volume is copied from an Amazon Machine Image (AMI), which can be chosen from a pre-populated list (including the ability to create your own AMI).

When an Amazon EC2 instance is Stopped, all EBS volumes remain attached to the instance. This allows the instance to be Started with the same configuration as when it was stopped.

When an Amazon EC2 instance is Terminated, EBS volumes might or might not be deleted, based upon the Delete on Termination setting of each volume:

By default, boot volumes are deleted when an instance is terminated. This is because the volume was originally just a copy of an AMI, so there is unlikely to be any important data on the volume. (Hint: Don't store data on a boot volume.)

Additional volumes default to "do not delete on termination", on the assumption that they contain data that should be retained. When the instance is terminated, these volumes will remain in an Available state, ready to be attached to another instance.

So, if you do not require any content on your remaining EBS volumes, simply delete them. In future, when launching instances, keep an eye on the Delete on Termination setting to make the clean-up process simpler.

Please note that the df -h command is only showing currently-attached volumes. It is not showing the volumes in Available state, since they are not visible to that instance. The concept of "Disk Space" typical refers to the space within an EBS volume, while "EBS Storage" refers to the volumes themselves. So, the 7GB of the volume that is used is related to that specific (boot) volume.

If you are running out of space on an EBS volume, see: Expanding the Storage Space of an EBS Volume on Linux. Expanding the volume involves:

• Creating a snapshot
• Creating a new (bigger) volume from the snapshot
• Swapping the disks (requiring a Stop/Start if you are swapping a boot volume)

How to check remaining space in storage device EC2

After you login using putty, enter command:

$ df -h

df - Disk filesystem
-h is human readable format

Full space on EC2

You can use "du" to find largest directories:

# du -mx / | sort -nr | head -10
1427    /
994     /usr
696     /usr/lib
429     /var
393     /usr/lib/x86_64-linux-gnu
346     /var/lib
174     /usr/share
159     /var/lib/snapd/seed/snaps
159     /var/lib/snapd/seed
159     /var/lib/snapd

Also you might try to reboot to force any open but deleted files to free space.

xvda1 is 100% full, What is it? how to fix?

That file, / is your root directory. If it's the only filesystem you see in df, then it's everything. You have a 1GB filesystem and it's 100% full. You can start to figure out how it's used like this:

sudo du -x / | sort -n | tail -40

You can then replace / with the paths that are taking up the most space. (They'll be at the end, thanks to the sort. The command may take awhile.)

Freeing some memory space on my Amazon Linux 2

To free memory you can do the following:

1. Check log files located in /var/log and empty any big log files.
2. Install and configure logrotate tool to make sure that logs are will not consume disk space specially if you are using any webserver "Nginx, Apache for example".
3. Remove unused packages and delete unused files.
4. Use find command as the following:
   find / -size +100M | xargs du -sch this command will list big files and show it's size.

---

Related Topics