Can Rails Migrations Be Used to Convert Data

Can Rails Migrations be used to convert data?

What you're trying to do is possible, and I would say the correct thing to do.

You need, though, to reload the column info for the model classes you're updating in the migration, so that Rails knows about the new columns. Try this:

def.self up
    add_column :users, :age_text, :string

    User.reset_column_information 

    users = User.find(:all)

    users.each do |u|
       u.age_text = convert_to_text(u.age)
       u.save
    end
end

On a separate note, please note that if your table is large, doing updates one by one will take a looong time.. Be careful with that.

Rails migration: only for schema change or also for updating data?

The short version is, since migrations are only for schema changes, you wouldn't want to use them to change actual data in the database.

The main issue is that your data-manipulating migration(s) might be ignored by other developers if they load the DB structuring using either rake db:schema:load or rake db:reset. Both of which merely load the latest version of the structure using the schema.rb file and do not touch the migrations.

As Nikita Singh also noted in the comments, I too would say the best method of changing row data is to implement a simple rake task that can be run as needed, independent of the migration structure. Or, for a first time installation, the seed.rb file is perfect to load initial system data.

Hope that rambling helps.

Update

Found some documentation in some "official" sources:

• Rails Guide for Migrations - Using Models in your Migrations. This section gives a description of a scenario in which data-manipulation in the migration files can cause problems for other developers.
• Rails Guide for Migrations - Migrations and Seed Data. Same document as above, doesn't really explain why it is bad to put seed or data manipulation in the migration, merely says to put all that in the seed.rd file.
• This SO answer. This person basically says the same thing I wrote above, except they provide a quote from the book Agile Web Development with Rails (3rd edition), partially written by David Heinemeier Hansson, creator of Rails. I won't copy the quote, as you can read it in that post, but I believe it gives you a better idea of why seed or data manipulation in migrations might be considered a bad practice.

How can I convert all migration files into a single file in Rails?

TL;DR

What you need isn't a consolidated set of migrations; it's a single schema file and an optional seeds.rb file. Rails generally maintains the schema automagically when you run migrations, so you already have most of what you should need with the possible exception of seed data as described below.

Use the Schema, Not Migrations

In general, you shouldn't be maintaining a large pool of migrations. Instead, you should periodically clear out your migrations, and use schema.rb or schema.sql to (re)create a database. The Rails guides specifically state:

There is no need (and it is error prone) to deploy a new instance of an app by replaying the entire migration history. It is much simpler and faster to just load into the database a description of the current schema...Because schema dumps are the authoritative source for your database schema, it is strongly recommended that you check them into source control.

You should therefore be using bin/rails db:schema:load rather than running migrations, or run the associated Rake task on older versions of Rails.

Data Migrations

While you can use migrations to fix or munge data related to a recent schema change, data migrations (if used at all) should be temporary artifacts. Data migrations are almost never idempotent, so you shouldn't be maintaining data migrations long-term. The guide says:

Some people use migrations to add data to the database...However, Rails has a 'seeds' feature that should be used for seeding a database with initial data. It's a really simple feature: just fill up db/seeds.rb with some Ruby code, and run rake db:seed...This is generally a much cleaner way to set up the database of a blank application.

Database seed data should be loaded with bin/rails db:seed (or the associated Rake task) rather than maintaining the data in migrations.

Rails migrations - change_column with type conversion

Since I'm using Postgres, I went with SQL solution for now.
Query used:

    execute 'ALTER TABLE "users" ALTER COLUMN "smoking" TYPE boolean USING CASE WHEN "flatshare"=\'true\' THEN \'t\'::boolean ELSE \'f\'::boolean END'

It works only if one has a field filled with true/false strings (such as default radio button collection helper with forced boolean type would generate)

When I run a schema migration before a data migration, with ActiveRecord, data does not properly update in DB

You're assuming after the first migration runs (change_column :users, :status, :string, default: User::Status::ACTIVE) you can still fetch the old values from the status column which is not the case. When you change the type of that column to string all the integer values are invalid so I suspect your database just changes all the invalid values to be "0" instead.

If I was told to make this change to an application that is heavily used in production, I would be roll out this change in a few separate pull requests/migrations. I'd create a whole new separate column, iterate through all the users, set the value of the new column depending on what the value in the old column is, and then delete the old column. This is a much safer way to make this change.

Rails: change column type, but keep data

A standard migration using the change_column method will convert integers to strings without any data loss. rake db:rollback will also do the reverse migration without error if required.

Here is the test migration I used to confirm this behaviour:

class ChangeAgeToString < ActiveRecord::Migration
  def self.up
    change_column :users, :age, :string
  end

  def self.down
    change_column :users, :age, :integer
  end
end

Rails Migration to convert string to integer?

Do not drop the column, it will clear the data.

You can however try

change_column :people, :company_id, :integer

and if all values in company_id can be converted to integer, it should be fine.

If that is not the case (ie not all string can be converted by default), then you can do it in two steps: 1) create a new column, then load the company_id in there after some conversion. 2) drop company_id then rename the new column.

You should be careful with both methods (more so for the second one) and you should probably do it first on a copy of the database, if you can.

Migrating DATA - not just schema, Rails

Best practice is: don't use models in migrations. Migrations change the way AR maps, so do not use them at all. Do it all with SQL. This way it will always work.

This:

User.all.each do |user|
  user.applied_at = user.partner_application_at
  user.save
end

I would do like this

update "UPDATE users SET applied_at=partner_application_at"

---

Related Topics