Filtering Sensitive Data with Vcr

Filtering sensitive data with VCR

VCR.configure do |c|
  c.filter_sensitive_data("<SOMESITE_PASSWORD>") do
    ENV['SOMESITE_PASSWORD']
    # or $credentials['somesite']['password'] or whatever
  end
end

Essentially, you give VCR a bit of placeholder text, and then the block needs to return the real password, reading it from whatever the canonical password "repository" is.

Note that the real password is only needed the first time, when the request is recorded; on subsequent runs, it can be a fake password (as long as it's the same fake password used by the code making the request).

Can't filter sensitive data with VCR

Well, looks like it's safe, if your key isn't there. To be sure, you might use regexp matcher to replace whole string, something like %r<#{keys['s3_key']:.*?=>. Bad news: there are no regexp filter_sensitive_data. Good news: you can use more low-level methods to implement that yourself.

That's current implementation of filter_sensitive_data

# @param placeholder [String] The placeholder string.
# @param tag [Symbol] Set this to apply this only to cassettes
#  with a matching tag; otherwise it will apply to every cassette.
# @yield block that determines what string to replace
# @yieldparam interaction [(optional) VCR::HTTPInteraction::HookAware] the HTTP interaction
# @yieldreturn the string to replace
def define_cassette_placeholder(placeholder, tag = nil, &block)
  before_record(tag) do |interaction|
    orig_text = call_block(block, interaction)
    log "before_record: replacing #{orig_text.inspect} with #{placeholder.inspect}"
    interaction.filter!(orig_text, placeholder)
  end

  before_playback(tag) do |interaction|
    orig_text = call_block(block, interaction)
    log "before_playback: replacing #{placeholder.inspect} with #{orig_text.inspect}"
    interaction.filter!(placeholder, orig_text)
  end
end
alias filter_sensitive_data define_cassette_placeholder

Source

Which leads us to these methods

  # Replaces a string in any part of the HTTP interaction (headers, request body,
  # response body, etc) with the given replacement text.
  #
  # @param [#to_s] text the text to replace
  # @param [#to_s] replacement_text the text to put in its place
  def filter!(text, replacement_text)
    text, replacement_text = text.to_s, replacement_text.to_s
    return self if [text, replacement_text].any? { |t| t.empty? }
    filter_object!(self, text, replacement_text)
  end

private

  def filter_object!(object, text, replacement_text)
    if object.respond_to?(:gsub)
      object.gsub!(text, replacement_text) if object.include?(text)
    elsif Hash === object
      filter_hash!(object, text, replacement_text)
    elsif object.respond_to?(:each)
      # This handles nested arrays and structs
      object.each { |o| filter_object!(o, text, replacement_text) }
    end

    object
  end

Source

Oh well, we might just try monkey patching this method:

Somewhere in your spec_helper:

class VCR::HTTPInteraction::HookAware
  def filter!(text, replacement_text)
    replacement_text = replacement_text.to_s unless replacement_text.is_a?(Regexp)
    text = text.to_s
    return self if [text, replacement_text].any? { |t| t.empty? }
    filter_object!(self, text, replacement_text)
  end
end

Of course, you can just opt out messing with the deep internals of alien library, and don't feel too paranoid knowing that some random alpha-numeric data is written to cassette near your token (but not including the latter).

After filtering sensitive data using VCR, re-running the spec fails with bad URI error

The filter_sensitive_data is a wrapper for two methods: before_record and before_playback. I was able to use those methods to find and replace usernames and passwords in the interaction - the first before writing to the YAML file, and the second before playing a cassette back when re-running a test.

This link: https://groups.google.com/forum/#!searchin/vcr-ruby/sensitive/vcr-ruby/uSm8HDBiWYw/OWIJk2_krVMJ, especially the comment from Myron Marston, provided a rough outline that I then modified to fit my specific API calls.

Filtering out JWT and Bearer tokens with VCR

I used filter_sensitive_data and came up with this:

VCR.configure do |config|
  config.filter_sensitive_data('<BEARER_TOKEN>') { |interaction|
    auths = interaction.request.headers['Authorization'].first
    if (match = auths.match /^Bearer\s+([^,\s]+)/ )
      match.captures.first
    end
  }
end

When I test, the auth header inside the cassette looks like:

Authorization:
- Bearer <BEARER_TOKEN>

Notable assumptions:

• An HTTP request should only contain a single auth header
• However, that header might contain multiple comma-separated auths
• The above code only captures an auth starting with 'Bearer'
• You could tweak it for other types other than 'Bearer'

---

Related Topics