Heroku Repo Size and Slug Size Increase with Each Deployment. Why

Heroku Slug Size After Multiple Deployments

This is probably somewhat related to Rail's attempt to avoid problems during deploys. To goal is to avoid downtime and/or errors during deploy. Your code isn't deployed instantly to all servers. Rails solution is to allow multiple versions of the assets to exist simultaneously. The typically process is:

%> rake assets:precompile  # To build new assets
%> rake assets:clean       # Remove old assets

Rails keeps 2 previous versions of the assets (by default). Depending on the application you may need old assets for a while. For example if you have a heavy JavaScript application that fetches resources once then is dynamically updated through XHR. The old code running in the client may refer to other old resources. Even with a non-client heavy app you may have several seconds where some nodes are referring to old assets and some are referring to new assets.

This would explain some of the reason public is larger on your old app. It essentially has three versions of all your assets while your new deploy only has one. It sounds like something must be messed up with your build environment though because even if all your assets occupy the 1.4MB in your new app, three times that size should only be about 5MB and not the 130MB in your old app.

In addition to the have three versions of your assets, there is also some accumulation in the tmp directory. The assets compiling process caches some information in the tmp directory. Rails has a rake tmp:cache:clear that you can periodically run to git rid of cruft there.

Heroku automatically runs rake assets:clean. So that should keep it to just three versions of the assets. But Heroku doesn't actually run rake tmp:cache:clear. Instead they have some custom code that deletes cache files until the cached data is under 50MB. I'm assuming they do this to keep as much cache info as possible while still capping things. Keeping as much cached data probably ensures asset compilation runs fast. This means your tmp directory will continue to grow until it reaches 50MB.

If your public/assets directory is growing beyond just 3 versions or 50MB in your tmp directory is too much then creating a new app is a decent way to clear things out. You could also create a custom slug. Just running rake assets:clean or rake tmp:cache:clear on a heroku console won't work. That will just clean things on a dyno and not your slug. So your cleaning will be thrown away when a new dyno is created.

Heroku: Compiled Slug Size is too large

It's possible that before you added the .slugignore file you had some large files added the git repo and now they are in the slug cache or as git refs. The git-repo plugin has commands to fix these problems:

$ heroku repo:gc -a appname

Will run git gc --aggressive on your repo.

$ heroku repo:purge_cache -a appname

This will delete the build cache and then you probably should run to rebuild the application.

$ heroku repo:rebuild -a appname

Heroku: Compiled Slug Size is too large Python

The maximum allowed slug size is 500MB. Slugs are an important aspect for heroku. When you git push to Heroku, your code is received by the slug compiler which transforms your repository into a slug.

First of all, lets determine what all files are taking up a considerate amount of space in your slug. To do that, fire up your heroku cli and enter / access your dyno by typing the following:

heroku run bash -a <appname>

Then, sort all the files present in the dyno by doing this:

du -ha --max-depth 1 /app | sort -hr

This would give you an idea about what files are taking up how much of a space.

Now, we can proceed to reduce the slug size. There are couple of ways through which you can reduce the slug size (assuming that you are using Heroku cli to deploy your application:

1. Using third party cloud storage: One of the popular ways in which you can reduce the slug size. Heroku supports Amazon S3 and Cloudinary among others. I prefer Amazon S3. Here is the documentation on how to setup S3 for Heroku. Move your large files and host them there. This would significantly reduce your slug size.

2. Using .slugignore : I can see that you are deploying some sort of ML application on heroku. Well, ML applications tend to be pretty heavy in size and frequently they cause problems during the deployment stage. Here is where .slugignore becomes very helpful. You can tell slug compiler to ignore some unnecessary files for slug compiler like images, test dataset, medias among many. More on how to use .slugignore can be found here in the official documentation

Heroku Slug Size exceed Limits More than 300m

As Heroku suggests:

• Inspect your slug with heroku run bash and by using commands such as ls and du
• Move large assets like PDFs or audio files to asset storage
• Remove unneeded dependencies and exclude unnecessary files via .slugignore

---

Related Topics