How to Get My Aws Lambda to Access Gems Stored in Vendor/Bundle

How to correctly load a gem extension in AWS Lambda

Do you have libpq.so.5 on your lib folder?

Your error is saying that did not find libpq.so.5 on the $PATH, in AWS Lambda the folder lib is automatically loaded on the path, thus, you just need to have this file there.

Executables created outside the lambda world do not run on Lambda, furthermore, you need to compile the executables by your own on a Lambda image. This is an example in how to do that:

Gemfile

source "https://rubygems.org"

gem "pg"
gem "mysql2"

handler.rb

require 'pg'
require 'mysql2'

def run(event:, context:)
  {
    postgres_client_version: PG.library_version,
    mysql_client_version: Mysql2::VERSION
  }
end

Dockerfile

FROM lambci/lambda:build-ruby2.5

RUN yum install -y postgresql postgresql-devel mysql mysql-devel
RUN gem update bundler

ADD Gemfile /var/task/Gemfile
ADD Gemfile.lock /var/task/Gemfile.lock

RUN bundle install --path /var/task/vendor/bundle --clean

This is going to build your image, then run it to generate the PG and MYSQL executables, then copy it to your lib folder.

build.sh

#!/bin/bash -x
set -e

rm -rf lib && rm -rf vendor && mkdir lib && mkdir vendor

docker build -t pg_mysql_layer -f Dockerfile .

CONTAINER=$(docker run -d pg_mysql_layer false)

docker cp \
    $CONTAINER:/var/task/vendor/ \
    ./

docker cp \
    $CONTAINER:/usr/lib64/libpq.so.5.5 \
    lib/libpq.so.5

docker cp \
    $CONTAINER:/usr/lib64/mysql/. \
    lib/

docker rm $CONTAINER

After running ./build.sh it is going to generate the folder lib and vendor with all you need, now you just need to deploy your lambda function.

To test locally you can run:
docker run --rm -it -v $PWD:/var/task -w /var/task lambci/lambda:ruby2.5 handler.run

It is going to return something similar to this:

REF: https://www.stevenringo.com/ruby-in-aws-lambda-with-postgresql-nokogiri/

REF: https://www.reddit.com/r/ruby/comments/a3e7a1/postgresql_on_aws_lambda_ruby/

Ruby bundle version doesn't match Ruby AWS Lambda function version

You're using Ruby 2.4.0 locally to bundle your Ruby application which makes Bundler (correctly) believe that you're targeting 2.4.0, which then results in the vendor/bundle/ruby/2.4.0/ folder being populated.

You should instead install and use Ruby 2.7.0 to sync your local Ruby version to the Lambda runtime you are targeting.

After verifying with ruby -v that you are indeed on 2.7.0, reinstall Bundler (gem install bundler) and then rerun the above commands.

You should then have the vendor/bundle/ruby/2.7.0/ folder populated instead after running bundle install, with the output of zip -r also showing that.

Ruby gem with native extensions not working on AWS Lambda

Running ldd on gems/oj-2.18.5/lib/oj/oj.so clarifies the first error. the problem is not that oj.so does not exist, but that libruby.so.2.5 does not exist. The second problem is that the current Ruby lambda has glibc version 2.17, where AWS-linux comes with glibc version 2.25.

The fundamental problem here is that you need to install your gems on a system identical to the one they will run on if you have native dependencies. The best way I've found to do this is using docker. https://github.com/lambci/docker-lambda has docker images of lambda installs.

For ruby, do the following:

1. Build the docker ruby image
2. Copy your gemfile into a pristine directory
3. From that directory, run: docker run -v "$PWD":/var/task --entrypoint bundle lambci/lambda-base:ruby2.5 install --path=/var/task

This will give you a folder called ruby with a version dependencies compatible with lambda.

If you are planning to use this output with lambda layers, keep in mind that the file structure produced by bunlde is ruby/2.5.0/... and it needs need to be ruby/gems/2.5.0/... before you upload it.

How to use a Gem in aws with Ruby?

Need to bundle non-aws gems into local vendor folder and then zip up those dependencies to be included, i.e.

bundle install --path vendor/bundle  # <-- Note using path

zip -r function.zip lambda_function.rb vendor/ # <-- vendor/ for the dependencies

aws lambda update-function-code --function-name myFunction \
--zip-file fileb://function.zip --region 'us-east-2' # <= your region

---

Related Topics