Directory Layout for Pure Ruby Project

Directory layout for pure Ruby project

Bundler includes the necessary infrastructure to generate a gem:

$ bundle gem --coc --mit --test=minitest --exe spider
Creating gem 'spider'...
MIT License enabled in config
Code of conduct enabled in config
      create  spider/Gemfile
      create  spider/lib/spider.rb
      create  spider/lib/spider/version.rb
      create  spider/spider.gemspec
      create  spider/Rakefile
      create  spider/README.md
      create  spider/bin/console
      create  spider/bin/setup
      create  spider/.gitignore
      create  spider/.travis.yml
      create  spider/test/test_helper.rb
      create  spider/test/spider_test.rb
      create  spider/LICENSE.txt
      create  spider/CODE_OF_CONDUCT.md
      create  spider/exe/spider
Initializing git repo in /Users/francois/Projects/spider
Gem 'spider' was successfully created. For more information on making a RubyGem visit https://bundler.io/guides/creating_gem.html

Then, in lib/, you create modules as needed:

lib/
  spider/
    base.rb
  crawler/
    base.rb
  spider.rb
    require "spider/base"
    require "crawler/base"

Read the manual page for bundle gem for details on the --coc, --exe and --mit options.

Ideal Ruby project structure

I think that is pretty much spot on. By default, Rubygems will add the lib directory to the loadpath, but you can push any directory you want onto that using the $: variable. i.e.

$:.push File.expand_path(File.dirname(__FILE__) + '/../surfcompstuff')

That means when you have say, surfer.rb in that dir, you can require "surfer" anywhere and the file will be found.

Also, as a convention, classes and singletons get a file and modules get a directory. For instance, if you had the LolCatz module and the LolCatz::Moar class that would look like:

lib/
  appname.rb
  lolcatz/
    moar.rb

That is why there is an lib/appname folder because most libraries are in the appname namespace.

Additionally, if you try running the command newgem --simple [projectname] that'll quickly generate a scaffold for you with just the bare essentials for a Ruby project (and by extension a Ruby Gem). There are other tools which do this, I know, but newgem is pretty common. I usually get rid of the TODO file and all the script stuff.

What's the standard file structure of a Ruby project?

There isn't necessarily any set way to define a Ruby project. But, generally RubyGems have this file structure:

.
├── bin
│   └── mygem
├── Gemfile
├── lib
│   ├── mygem
│   │   └── version.rb
│   └── mygem.rb
├── LICENSE
├── mygem.gemspec
├── Rakefile
└── README.md

Of course, there is also other directories and files in the project root, for tests, .ruby-version, etc.

What is the recommended folder structure for a Ruby command-line app with tests?

Unit Tests for Gems

If you're using MiniTest or similar, the expected place within gem foo would be foo/test. If you're using RSpec or minitest/spec, then you'd use foo/spec instead.

When you generate a new gem skeleton with Bundler it provides the following defaults:

$ bundle gem foo; tree foo

foo
├── Gemfile
├── README.md
├── Rakefile
├── bin
│   ├── console
│   └── setup
├── foo.gemspec
├── lib
│   ├── foo
│   │   └── version.rb
│   └── foo.rb
└── spec
    ├── foo_spec.rb
    └── spec_helper.rb

4 directories, 10 files

The default Rakefile is set up to run RSpec, but you can certainly implement the Rakefile's default task and your tests any way you'd like.

Starting a Ruby project: github + build tool

Jeweler
Rake

And here an example: Create your first ruby gem and release it to gemcutter

As for github, they stopped direct gem building but still you can host your source there easily.

To distribute your application as executable have a look at RubyScript2Exe or OCRA

How do I set up a basic Ruby project?

To get a good start, you can use the bundle gem command and rspec --init.

~/code $ bundle gem my_lib
      create  my_lib/Gemfile
      create  my_lib/Rakefile
      create  my_lib/LICENSE.txt
      create  my_lib/README.md
      create  my_lib/.gitignore
      create  my_lib/my_lib.gemspec
      create  my_lib/lib/my_lib.rb
      create  my_lib/lib/my_lib/version.rb
Initializating git repo in /Users/john/code/my_lib
~/code $ cd my_lib/
~/code/my_lib $ git commit -m "Empty project"
~/code/my_lib $ rspec --init
The --configure option no longer needs any arguments, so true was ignored.
  create   spec/spec_helper.rb
  create   .rspec

code goes in lib
specs go in spec
test data or documents go in spec/fixtures/
Require all your ruby files in lib/my_lib.rb. You can define your exceptions in that file, too, or in their own files -- according to your own preference.
C source files go in ext/my_lib
shell scripts and executables go in bin

When in doubt, just look at how other gems are laid out.

Further information:

You should add rspec as a development dependency in your gemspec to make things easier for other developers

Edit my_lib.gemspec, adding gem.add_development_dependency 'rspec' and gem.add_development_dependency 'rake' near the bottom.
Add Bundler.setup and require 'my_lib' to the top of spec/spec_helper.rb to ensure your gem dependencies are loaded when you run your specs.
Add require "rspec/core/rake_task" and task :default => :spec to your Rakefile, so that running rake will run your specs.

While you're working on your newest creation, guard-rspec can save you time and hassle by automatically running your specs as files change, alerting you to spec failures.

~/code/my_lib $ git add spec/spec_helper.rb
~/code/my_lib $ git commit -am "Add RSpec"
~/code/my_lib $ vim my_lib.gemspec # add guard development dependency
~/code/my_lib $ bundle
~/code/my_lib $ bundle exec guard init
~/code/my_lib $ vim Guardfile # Remove the sections below the top one
~/code/my_lib $ git add Guardfile
~/code/my_lib $ git commit -am "Add Guard"

After you're happy with your creation, push it up to github

# create a github repository for your gem, then push it up
~/code/my_lib $ curl -u myusername https://api.github.com/user/repos -d '{"name":"my_lib"}' 
~/code/my_lib $ git remote add origin git@github.com:myusername/my_lib.git
~/code/my_lib $ git push

Then, when you're ready to release your gem on Rubygems.org, run rake release, which will walk you through the steps.

~/code/my_lib $ rake release

Further References

The Rubygems patterns guide (and home page), from Matheus Moreira's answer. They're really great references
How I Start by Steve Klabnik
Exercise 46: A Project Skeleton from Zed Shaw's Learn Ruby The Hard Way
New Gem with Bundler video on Railscasts
docs

Ruby Packaging Ecosystem As Python Terminolgies

RVM is similar to virtualenv also checkout rbenv (perhaps more like virtualenv)

Bundler is for packaging dependencies for development and deployment, it works like setup.py and pip (I haven't used pip, it seems to have some features of rubygems and Bundler)

Bundler's Gemfile is similar to pip's requirement file

Bundler will install dependencies in your development directory, and package them for deployment.

Directory layouts tend to look like this:

/ -
  lib - classes / modules etc.
  bin - executables things you want on $PATH
  test - unit tests

Jeweler is a good tool for setting up, maintaining and releasing gems.

EDIT:

Here are some other resources:

Some links on Ruby layout:

Directory layout for pure Ruby project
Ideal ruby project structure
http://www.ruby-forum.com/topic/213637

Some for Python:

What is the best project structure for a Python application?
Initialize project layout in python?

Here's one making a comparison of tools:

http://news.ycombinator.com/item?id=2513005

Making a Ruby project that is not a gem (or some other kind)

You have to actually try to build a gem so this is easy!

to use bundler without Rails, a gem, whatever just create a directory

  mkdir my-non-gem-project
  cd my-non-gem-project

install bundler

  gem install bundler

and initialize your Gemfile

  bundle init

that will create a Gemfile for you and you can add to it and run bundle to install the dependencies from it

The simplest way to use bundler in your project would then be to open your main app file and add

require 'bundler/setup'
Bundler.require

This will require all of the gems you have in your Gemfile in the file this is added to. I am pretty sure that this file must be in the same directory as your Gemfile. More information here

Have fun with your Ruby project!

CarrierWave folder structure best practices

The normal folder structure should support hat very well, the main thing is to avoid collisions and make sure that association you use for images across your site is consistent. For these reasons you would not want to just store all of your images in one folder obviously.

But I'm thinking it is better to perhaps include the user_id in the folder name?

I would not recommend this, it may cause more problems than its worth. Using the models class, and id as you are doing should be plenty sufficient and also the uploader may have problems mapping the image for example when a user who is not the uploader tries to view the image.

Is it better to move this folder outside of the public/uploads folder?

Since you are using Amazon S3 your images are not actually stored in public/uploads of your project if that is where you are referring to. That should only be the temporary files that Carrierwave uses to while uploading/resizing the files. If you are worried about space/security you may want to take a look at this and adapt it to your needs if necessary.

Shouldn't I used different folders for the dev, production and testing environments?

You can use different folders if you like:

def store_dir
  "#{Rails.env}/uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end

Or use different buckets on Amazon S3 (recommended):

In config/initializers/fog.rb something like:

CarrierWave.configure do |config|
  config.storage = :fog
  config.fog_credentials = {
    :provider              => <provider>,
    :aws_access_key_id     => <id>,
    :aws_secret_access_key => <key>
  }
  if Rails.env == "production"
    config.fog_directory = 'production'
  elsif Rails.env == "development"
    config.fog_directory = 'development'
  elsif ...
    config.fog_directory = '...'
  end
end

Or you could do something similar in the separate environment files.

I'd like to override the store_dir in the best way for a site with thousands of users uploading around 15 files per profile (stored as full size, profile and thumbnail sizes).

This storage dir structure should work just fine, as far as resizing you should check out the guide.

Update:

For the file name I would definitely recommend changing it to a random string so that you can avoid any potential naming collisions within the folder itself. It is not unlikely that someone could upload me.jpg for 2 different files. That being said this is how I do it.

Within your_uploader.rb

add a filename method that will randomize the current files name.

 def filename
   random_token = Digest::SHA2.hexdigest("#{Time.now.utc}--#{model.id.to_s}").first(20)
   ivar = "@#{mounted_as}_secure_token"
   token = model.instance_variable_get(ivar)
   token ||= model.instance_variable_set(ivar, random_token)
   "#{token}.jpg" if original_filename
  end

This particular arrangement may be a bit overkill but it proves sufficient for me.

Hope this helps!

Determining the gem's list of files for the specification

With Rake

The easiest solution depending on rake to list all files from a directory, but exclude everything in the .gitignore file:

require 'rake/file_list'
Rake::FileList['**/*'].exclude(*File.read('.gitignore').split)

RubyGems

Official rubygems solution, list and exclude manually:

require 'rake'
spec.files = FileList['lib/*.rb',
                      'bin/*',
                      '[A-Z]*',
                      'test/*'].to_a

# or without Rake...
spec.files = Dir['lib/*.rb'] + Dir['bin/*']
spec.files += Dir['[A-Z]*'] + Dir['test/**/*']
spec.files.reject! { |fn| fn.include? "CVS" }

Bundler

Bundler solution, list manually:

s.files = Dir.glob("{lib,exe}/**/*", File::FNM_DOTMATCH).reject {|f| File.directory?(f) }

Note: rejecting directories is useless as gem will ignore them by default.

Vagrant

Vagrant solution to mimic git ls-files and taking care of .gitignore in pure ruby:

  # The following block of code determines the files that should be included
  # in the gem. It does this by reading all the files in the directory where
  # this gemspec is, and parsing out the ignored files from the gitignore.
  # Note that the entire gitignore(5) syntax is not supported, specifically
  # the "!" syntax, but it should mostly work correctly.
  root_path      = File.dirname(__FILE__)
  all_files      = Dir.chdir(root_path) { Dir.glob("**/{*,.*}") }
  all_files.reject! { |file| [".", ".."].include?(File.basename(file)) }
  all_files.reject! { |file| file.start_with?("website/") }
  all_files.reject! { |file| file.start_with?("test/") }
  gitignore_path = File.join(root_path, ".gitignore")
  gitignore      = File.readlines(gitignore_path)
  gitignore.map!    { |line| line.chomp.strip }
  gitignore.reject! { |line| line.empty? || line =~ /^(#|!)/ }

  unignored_files = all_files.reject do |file|
    # Ignore any directories, the gemspec only cares about files
    next true if File.directory?(file)

    # Ignore any paths that match anything in the gitignore. We do
    # two tests here:
    #
    #   - First, test to see if the entire path matches the gitignore.
    #   - Second, match if the basename does, this makes it so that things
    #     like '.DS_Store' will match sub-directories too (same behavior
    #     as git).
    #
    gitignore.any? do |ignore|
      File.fnmatch(ignore, file, File::FNM_PATHNAME) ||
        File.fnmatch(ignore, File.basename(file), File::FNM_PATHNAME)
    end
  end

Pathspec

Using pathspec gem Match Path Specifications, such as .gitignore, in Ruby!

See https://github.com/highb/pathspec-ruby

References

Ref:
Bundler
Vagrant
RubyGems
Rake easy solution

Directory Layout for Pure Ruby Project