How to Access a Gemified Padrino Apps Model from Outside (Not in Controller, But E.G. a Standalone Script)

How to access a gemified Padrino Apps Model from outside (not in controller, but e.g. a standalone script)

Sorry to hear you're having trouble with this. It's good that you brought it up though because I've been trying to put my thoughts around the subject for a while now and this pushed me into it :). I've prepared a repo for you explaining how to do it with what we have now in Padrino.

The README (which I'm pasting afterwards), explains the reasoning behind it and puts some questions up for us to think about the way we've implemented them. I'd love to hear your thoughts about it :).

Gemified apps in Padrino

This repo intends to answer
How to access Padrino model and database in a “standalon” (bin/) script? and
How to access a gemified Padrino Apps Model from other gem that requires that App.

The issue

In short, there are two issues of the similar nature, both related to models defined in the gemified app:

they need to be accessed from another gems/projects;
they need to be accessed from the gemified app's bin, doing something else other than starting
the Padrino server.

The example

First there's gemified-app. That's a Padrino app that is gemified. It also contains a model
called SomeModel that has one field called property.

Then there's access-gemified-app-without-padrino; a ruby script that loads the gemified app to
access the model.

Finally, there's another-app which is a regular Padrino app that just loads gemified-app to use
its model (SomeModel).

Problems with the current Padrino setup

Creating an app with padrino g project gemified-app --orm sequel --gem --tiny will give you the
following gemspec:

# -*- encoding: utf-8 -*-
require File.expand_path('../lib/gemified-app/version', __FILE__)

Gem::Specification.new do |gem|
  gem.authors       = ["Darío Javier Cravero"]
  gem.email         = ["dario@uxtemple.com"]
  gem.description   = %q{Padrino gemified app example}
  gem.summary       = %q{Padrino gemified app example}
  gem.homepage      = ""

  gem.files         = `git ls-files`.split($\)
  gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
  gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
  gem.name          = "gemified-app"
  gem.require_paths = ["lib", "app"]
  gem.version       = GemifiedApp::VERSION

  gem.add_dependency 'padrino-core'
end

The key points are gem.require_paths = ["lib", "app"] and gem.add_dependency 'padrino-core'.

gem.require_paths = ["lib", "app"] explains why models/some_model.rb isn't available when we
load the gem somewhere else. It simple isn't added to $LOAD_PATH :(.

gem.add_dependency 'padrino-core' hints us that something might be missing later on. What happens
with dependencies like the ORM or the renderer? Should we load those? I reckon that it's a matter
of what you want to achieve but I'd say that most times yes.

Our gemified app dependencies are still listed in our Gemfile which will only be added in the
current scope and not in any gems requiring our gemified-app gem.

A first attempt at solving this

For this to work there are two things we should do:

Add 'models' to gem.require_paths = ["lib", "app"] so that it becomes:
gem.require_paths = ["lib", "app", "models"].
That will make sure that anything inside the gemified-app/models directory is included in your
gem.

To make it easier to test this, we'll use bundler and in our access-gemified-app-without-padrino
test script we'll add a Gemfile that looks like this:

source 'https://rubygems.org'

gem 'gemified-app', path: '../gemified-app'
gem 'pry'

Now in your new app, go to the REPL bundle exec pry and try to require 'gemified-app'.
Then try SomeModel.all. It will fail. Why? Because you didn't require 'some_model'.

It will still not work if you do that though. Why? Because none of the model's dependencies,
i.e. sequel and sqlite3 (not a direct dependency but it is through the connection) are loaded.

Here you have two choices: you load them manually on your Gemfile or you define them as
dependencies on gemified-app.gemspec.
I regard the latter one as a better choice since you're already including the model and you're
expecting its dependencies to come with it. It would like this:

# gemified-app/gemified-app.gemspec

  # ...

  gem.add_dependency 'padrino-core'
  gem.add_dependency 'padrino-helpers'
  gem.add_dependency 'slim'
  gem.add_dependency 'sqlite3'
  gem.add_dependency 'sequel'
  gem.add_development_dependency 'rake'

  # ...

# gemified-app/Gemfile
source 'https://rubygems.org'

# Distribute your app as a gem
gemspec

You would have to explicitly include all the gems you will need. This may seem cumbersome but in
all fairness it gives you a greater understanding of what your app needs. Eventually you will
realise you don't even need bundler and the Gemfile :).

Alright, so, go ahead launch your REPL and type require 'gemified-app' and require 'some_model'.
Then try SomeModel.all. And... It will fail :(. Why? Because Sequel::Base isn't defined. Now you might be wondering:
what happened to the reference to sequel I put in my gemified-app.gemspec? Well, it's just that:
a reference and it won't require the gem for you.
This won't happen with Padrino either because we're using

require 'rubygems' unless defined?(Gem)
require 'bundler/setup'
Bundler.require(:default, RACK_ENV)

in our config/boot.rb and that only loads required gems on our Gemfile.

So the question is... Should we load that manually? And if so, where?

Well, since this is a gem itself, I believe that the best place to do so would be in lib/gemified-app.rb.
Loading all the gems needed will make this file look like:

require 'padrino-core'
require 'padrino-helpers'
require 'slim'
require 'sqlite3'
require 'sequel'

module GemifiedApp
  extend Padrino::Module
  gem! "gemified-app"
end

Alright, so we're all set... Back to the REPL, do your requires

require 'gemified-app'
require 'some_model'

and try SomeModel.all. And... It will fail :(. Again! :/ Why? Because there's no connection to the
database. Padrino was loading this for us through config/database.rb.

Another question arises... Should we include config/database.rb in the gem too?
The way I see it, we shouldn't. The way I see it, the database connection is something every app
should locally define as it may contain specific credentials to access it or stuff like that.
Our sample, access-gemified-app-without-padrino/do-somethin.rb script will then look like this:

require 'gemified-app'

Sequel::Model.plugin(:schema)
Sequel::Model.raise_on_save_failure = false # Do not throw exceptions on failure
Sequel::Model.db = Sequel.connect("sqlite:///" + File.expand_path('../../gemified-app/db/gemified_app_development.db', __FILE__), :loggers => [logger])

require 'some_model'

SomeModel.all.each do |model|
  puts %Q[#{model.id}: #{model.property}]
end

Yes, the connection code is pretty much the same than our Padrino app and we're reusing its database
for this example.

That was some ride :) but we finally made it. See the sample apps in the repo for some working
examples.

require `some_model` :/

I don't know you but I don't like that at all. Having to do something like that means that I
really have to pick my models' names very carefully not to clash with anything I may want to use
in the future.
I reckon that modules are the answer to it but that's the current state of affairs. See the
conclusion for more on this.

An alternative approach

Separate your model layer into its own gem and require it from your (gemified or not) Padrino app.
This might probably be the cleanest as you can isolate tests for your models and even create
different models for different situations that may or may not use the same database underneath.

It could also encapsulate all of the connection details.

Conclusion

I think we should review Padrino's approach to gemified apps.

Should we use the gemspec instead of the Gemfile for hard dependencies?

Should we namespace the models (I know we had some issues in the past with this)?

Should we teach users to do explicit requires in their gems or to inspect the dependecies and
require them for them?

Should we teach our users how to load their dependencies and be more reponsible about it? At the end
of the day, if they went the gemified app route they are clearly much more proficient in Ruby and
should be aware of this kind of stuff.

Thoughts? :)

How to access a gemified Padrino Apps Model from outside (not in controller, but e.g. a standalone script)

Gemified apps in Padrino

This repo intends to answer
How to access Padrino model and database in a “standalon” (bin/) script? and
How to access a gemified Padrino Apps Model from other gem that requires that App.

The issue

In short, there are two issues of the similar nature, both related to models defined in the gemified app:

they need to be accessed from another gems/projects;
they need to be accessed from the gemified app's bin, doing something else other than starting
the Padrino server.

The example

First there's gemified-app. That's a Padrino app that is gemified. It also contains a model
called SomeModel that has one field called property.

Then there's access-gemified-app-without-padrino; a ruby script that loads the gemified app to
access the model.

Finally, there's another-app which is a regular Padrino app that just loads gemified-app to use
its model (SomeModel).

Problems with the current Padrino setup

Creating an app with padrino g project gemified-app --orm sequel --gem --tiny will give you the
following gemspec:

# -*- encoding: utf-8 -*-
require File.expand_path('../lib/gemified-app/version', __FILE__)

Gem::Specification.new do |gem|
  gem.authors       = ["Darío Javier Cravero"]
  gem.email         = ["dario@uxtemple.com"]
  gem.description   = %q{Padrino gemified app example}
  gem.summary       = %q{Padrino gemified app example}
  gem.homepage      = ""

  gem.files         = `git ls-files`.split($\)
  gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
  gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
  gem.name          = "gemified-app"
  gem.require_paths = ["lib", "app"]
  gem.version       = GemifiedApp::VERSION

  gem.add_dependency 'padrino-core'
end

The key points are gem.require_paths = ["lib", "app"] and gem.add_dependency 'padrino-core'.

gem.require_paths = ["lib", "app"] explains why models/some_model.rb isn't available when we
load the gem somewhere else. It simple isn't added to $LOAD_PATH :(.

Our gemified app dependencies are still listed in our Gemfile which will only be added in the
current scope and not in any gems requiring our gemified-app gem.

A first attempt at solving this

For this to work there are two things we should do:

To make it easier to test this, we'll use bundler and in our access-gemified-app-without-padrino
test script we'll add a Gemfile that looks like this:

source 'https://rubygems.org'

gem 'gemified-app', path: '../gemified-app'
gem 'pry'

Now in your new app, go to the REPL bundle exec pry and try to require 'gemified-app'.
Then try SomeModel.all. It will fail. Why? Because you didn't require 'some_model'.

It will still not work if you do that though. Why? Because none of the model's dependencies,
i.e. sequel and sqlite3 (not a direct dependency but it is through the connection) are loaded.

# gemified-app/gemified-app.gemspec

  # ...

  gem.add_dependency 'padrino-core'
  gem.add_dependency 'padrino-helpers'
  gem.add_dependency 'slim'
  gem.add_dependency 'sqlite3'
  gem.add_dependency 'sequel'
  gem.add_development_dependency 'rake'

  # ...

# gemified-app/Gemfile
source 'https://rubygems.org'

# Distribute your app as a gem
gemspec

require 'rubygems' unless defined?(Gem)
require 'bundler/setup'
Bundler.require(:default, RACK_ENV)

in our config/boot.rb and that only loads required gems on our Gemfile.

So the question is... Should we load that manually? And if so, where?

Well, since this is a gem itself, I believe that the best place to do so would be in lib/gemified-app.rb.
Loading all the gems needed will make this file look like:

require 'padrino-core'
require 'padrino-helpers'
require 'slim'
require 'sqlite3'
require 'sequel'

module GemifiedApp
  extend Padrino::Module
  gem! "gemified-app"
end

Alright, so we're all set... Back to the REPL, do your requires

require 'gemified-app'
require 'some_model'

and try SomeModel.all. And... It will fail :(. Again! :/ Why? Because there's no connection to the
database. Padrino was loading this for us through config/database.rb.

require 'gemified-app'

Sequel::Model.plugin(:schema)
Sequel::Model.raise_on_save_failure = false # Do not throw exceptions on failure
Sequel::Model.db = Sequel.connect("sqlite:///" + File.expand_path('../../gemified-app/db/gemified_app_development.db', __FILE__), :loggers => [logger])

require 'some_model'

SomeModel.all.each do |model|
  puts %Q[#{model.id}: #{model.property}]
end

Yes, the connection code is pretty much the same than our Padrino app and we're reusing its database
for this example.

That was some ride :) but we finally made it. See the sample apps in the repo for some working
examples.

require `some_model` :/

An alternative approach

It could also encapsulate all of the connection details.

Conclusion

I think we should review Padrino's approach to gemified apps.

Should we use the gemspec instead of the Gemfile for hard dependencies?

Should we namespace the models (I know we had some issues in the past with this)?

Should we teach users to do explicit requires in their gems or to inspect the dependecies and
require them for them?

Thoughts? :)

Read-only web apps with Rails/Sinatra

Both of those are workable and I have employed both in the past, but I'd go with the API approach.

Quick disclaimer: one thing that's not clear is how different these apps are in function. For example, I can imagine the old one being a CRUD app that works on individual records and the new one being a reporting app that does big complicated aggregation queries. That makes the shared DB (maybe) more attractive because the overlap in how you access the data is so small. I'm assuming below that's not the case.

Anyway, the API approach. First, the bad:

One more dependency (the old app). When it breaks, it takes down both apps.
One more hop to get data, so higher latency.
Working with existing code is less fun than writing new code. Just is.

But on the other hand, the good:

Much more resilient to schema changes. Your "old" app's API can have tests, and you can muck with the database to your heart's content (in the context of the old app) and just keep your API to its spec. Your new app won't know the difference, which is good. Abstraction FTW. This the opposite side of the "one more dependency" coin.
Same point, but from different angle: in the we-share-the-database approach, your schema + all of SQL is effectively your API, and it has two clients, the old app and the new. Unless your two apps are doing very different things with the same data, there's no way that's the best API. It's too poorly defined.
The DB admin/instrumentation is better. Let's say you mess up some query and hose your database. Which app was it? Where are these queries coming from? Basically, the fewer things that can interact with your DB, the better. Related: optimize your read queries in one place, not two.
If you used RESTful routes in your existing app for the non-API actions, I'm guessing your API needs will have a huge overlap with your existing controller code. It may be a matter of just converting your data to JSON instead of passing it to a view. Rails makes it very easy to use an action to respond to both API and user-driven requests. So that's a big DRY win if it's applicable.
What happens if you find out you do want some writability in your new app? Or at least access to some field your old app doesn't care about (maybe you added it with a script)? In the shared DB approach, it's just gross. With the other, it's just a matter of extending the API a bit.

Basically, the only way I'd go for the shared DB approach is that I hated the old code and wanted to start fresh. That's understandable (and I've done exactly that), but it's not the architecturally soundest option.

A third option to consider is sharing code between the two apps. For example, you could gem up the model code. Now your API is really some Ruby classes that know how to talk to your database. Going even further, you could write a Sinatra app and mount it inside of the existing Rails app and reuse big sections it. Then just work out the routing so that they look like separate apps to the outside world. Whether that's practical obviously depends on your specifics.

In terms of specific technologies, both Sinatra and Rails are fine choices. I tend towards Rails for bigger projects and Sinatra for smaller ones, but that's just me. Do what feels good.

Using Sinatra for larger projects via multiple files

Here is a basic template for Sinatra apps that I use. (My larger apps have 200+ files broken out like this, not counting vendor'd gems, covering 75-100 explicit routes. Some of these routes are Regexp routes covering an additional 50+ route patterns.) When using Thin, you run an app like this using:

thin -R config.ru start

Edit: I'm now maintaining my own Monk skeleton based on the below called Riblits. To use it to copy my template as the basis for your own projects:

# Before creating your project
monk add riblits git://github.com/Phrogz/riblits.git

# Inside your empty project directory
monk init -s riblits

File Layout:


config.ru
app.rb
helpers/
  init.rb
  partials.rb
models/
  init.rb
  user.rb
routes/
  init.rb
  login.rb
  main.rb
views/
  layout.haml
  login.haml
  main.haml

config.ru

root = ::File.dirname(__FILE__)
require ::File.join( root, 'app' )
run MyApp.new

app.rb

# encoding: utf-8
require 'sinatra'
require 'haml'

class MyApp < Sinatra::Application
  enable :sessions

  configure :production do
    set :haml, { :ugly=>true }
    set :clean_trace, true
  end

  configure :development do
    # ...
  end

  helpers do
    include Rack::Utils
    alias_method :h, :escape_html
  end
end

require_relative 'models/init'
require_relative 'helpers/init'
require_relative 'routes/init'

helpers/init.rb

# encoding: utf-8
require_relative 'partials'
MyApp.helpers PartialPartials

require_relative 'nicebytes'
MyApp.helpers NiceBytes

helpers/partials.rb

# encoding: utf-8
module PartialPartials
  def spoof_request(uri,env_modifications={})
    call(env.merge("PATH_INFO" => uri).merge(env_modifications)).last.join
  end

  def partial( page, variables={} )
    haml page, {layout:false}, variables
  end
end

helpers/nicebytes.rb

# encoding: utf-8
module NiceBytes
  K = 2.0**10
  M = 2.0**20
  G = 2.0**30
  T = 2.0**40
  def nice_bytes( bytes, max_digits=3 )
    value, suffix, precision = case bytes
      when 0...K
        [ bytes, 'B', 0 ]
      else
        value, suffix = case bytes
          when K...M then [ bytes / K, 'kiB' ]
          when M...G then [ bytes / M, 'MiB' ]
          when G...T then [ bytes / G, 'GiB' ]
          else            [ bytes / T, 'TiB' ]
        end
        used_digits = case value
          when   0...10   then 1
          when  10...100  then 2
          when 100...1000 then 3
          else 4
        end
        leftover_digits = max_digits - used_digits
        [ value, suffix, leftover_digits > 0 ? leftover_digits : 0 ]
    end
    "%.#{precision}f#{suffix}" % value
  end
  module_function :nice_bytes  # Allow NiceBytes.nice_bytes outside of Sinatra
end

models/init.rb

# encoding: utf-8
require 'sequel'
DB = Sequel.postgres 'dbname', user:'bduser', password:'dbpass', host:'localhost'
DB << "SET CLIENT_ENCODING TO 'UTF8';"

require_relative 'users'

models/user.rb

# encoding: utf-8
class User < Sequel::Model
  # ...
end

routes/init.rb

# encoding: utf-8
require_relative 'login'
require_relative 'main'

routes/login.rb

# encoding: utf-8
class MyApp < Sinatra::Application
  get "/login" do
    @title  = "Login"
    haml :login
  end

  post "/login" do
    # Define your own check_login
    if user = check_login
      session[ :user ] = user.pk
      redirect '/'
    else
      redirect '/login'
    end
  end

  get "/logout" do
    session[:user] = session[:pass] = nil
    redirect '/'
  end
end

routes/main.rb

# encoding: utf-8
class MyApp < Sinatra::Application
  get "/" do
    @title = "Welcome to MyApp"        
    haml :main
  end
end

views/layout.haml

!!! XML
!!! 1.1
%html(xmlns="http://www.w3.org/1999/xhtml")
  %head
    %title= @title
    %link(rel="icon" type="image/png" href="/favicon.png")
    %meta(http-equiv="X-UA-Compatible" content="IE=8")
    %meta(http-equiv="Content-Script-Type" content="text/javascript" )
    %meta(http-equiv="Content-Style-Type" content="text/css" )
    %meta(http-equiv="Content-Type" content="text/html; charset=utf-8" )
    %meta(http-equiv="expires" content="0" )
    %meta(name="author" content="MeWho")
  %body{id:@action}
    %h1= @title
    #content= yield

How to Access a Gemified Padrino Apps Model from Outside (Not in Controller, But E.G. a Standalone Script)

How to access a gemified Padrino Apps Model from outside (not in controller, but e.g. a standalone script)

Gemified apps in Padrino

The issue

The example

Problems with the current Padrino setup

A first attempt at solving this

require `some_model` :/

An alternative approach

Conclusion

How to access a gemified Padrino Apps Model from outside (not in controller, but e.g. a standalone script)

Gemified apps in Padrino

The issue

The example

Problems with the current Padrino setup

A first attempt at solving this

require `some_model` :/

An alternative approach

Conclusion

Read-only web apps with Rails/Sinatra

Using Sinatra for larger projects via multiple files

Related Topics

Leave a reply

How to access a gemified Padrino Apps Model from outside (not in controller, but e.g. a standalone script)

Gemified apps in Padrino

The issue

The example

Problems with the current Padrino setup

A first attempt at solving this

require some_model :/

An alternative approach

Conclusion

How to access a gemified Padrino Apps Model from outside (not in controller, but e.g. a standalone script)

Gemified apps in Padrino

The issue

The example

Problems with the current Padrino setup

A first attempt at solving this

require some_model :/

An alternative approach

Conclusion

Read-only web apps with Rails/Sinatra

Using Sinatra for larger projects via multiple files

Related Topics

Leave a reply

require `some_model` :/

require `some_model` :/