Filter Rails JSON input with route constraints

Following the recent YAML parsing vulnerabilities in Rails, I decided to act on an idea I had a few months ago: using route constraints to define strict API contracts in Rails.

Sadly, it does not protect against the YAML parsing problem (and the probable similar vulnerabilities we will see in the following months), because the request is interpreted before going through the route constraints. But it can protect from the mass assignment vulnerability., and probably some SQL injections.

Here is the idea: Rails 3 introduced the route constraints, a way to execute fonctions on the request before it is passed to the controllers. By combining it with the json-schema gem, we can filter the JSON input quite easily.

For the following example data:

{"echo" : "blah", "nb" : 1, "data": [1, 2, 3]}

We can define the following schema:

{
    "type": "object",
    "$schema": "http://json-schema.org/draft-03/schema",
    "id": "#",
    "required": false,
    "additionalProperties": false,
    "properties": {
        "data": {
            "type": "array",
            "id": "data",
            "required": false,
            "items": {
                "type": "number",
                "id": "0",
                "required": false
            }
        },
        "echo": {
            "type": "string",
            "id": "echo",
            "required": false
        },
        "nb": {
            "type": "number",
            "id": "nb",
            "required": false
        }
    }
}

(Use the JSON schema generator to create your own)

Save this schema to “data.schema” and add “json-schema” to your Gemfile. You will then be able to filter inputs with code like the following “config/routes.rb”:

require "json-schema"
class LolJSONConstraint
  def matches?(request)
    if(request.headers["CONTENT_TYPE"] == "application/json")
      JSON::Validator.validate("data.schema", request.headers["action_dispatch.request.request_parameters"])
    end
  end
end

Yamlvuln::Application.routes.draw do
  resources :posts, :constraints => LolJSONConstraint.new
end

The constraint will load the schema, and apply it to the incoming data, and return a 404 error if the JSON is invalid. The “additionalProperties” set to false in the schema is required to refuse the properties you didn’t define and protect the application from mass assignment.

If I tried, for example, to send the following JSON to the application, there would be an error:

{"echo" : "blah", "nb" : "UNION ALL SELECT LOAD_FILE(CHAR(34,47,101,116,99,47,112,97,115,115,119,100,34))", "data": [1, 2, 3]}

As I said before, it is not safe against the YAML parsing vulnerability. Also, I did not really test the performance of this. But it is still a nice and easy solution for API filtering.

About these ads

Jtalk on Rails: editing Javascript in my browser

If you don’t know JTalk yet, you’re missing something. It’s an awesome piece of work: a Smalltalk to Javascript compiler and a Smalltalk editor running in Javascript, IN YOUR BROWSERGo check it out, now!

Now that you’ve played a bit with JTalk, let’s get started.

If you’re like me, you’re a bit annoyed by WebDAV, the proposed solution to save changes to disk. And if you’re like me, you would like to use Jtalk with Rails, and because you’re a lazy ass like me, you use WEBrick instead of Apache for your development.

Let’s hack something up to replace WebDAV!

Create a Rails application

rails new jtalkonrails
cd jtalkonrails
bundle install
rm public/index.html
rails generate controller home index
printf "Jtalkonrails::Application.routes.draw do\n  root :to => \"home#index\"\nend\n" > config/routes.rb

(I should really make a script out of all my rails initialization commands, one of these days…)

Add Jtalk to your application

cd public/
wget http://github.com/NicolasPetton/jtalk/tarball/master --no-check-certificate
tar zxvf master
cp -R NicolasPetton-jtalk-20cd63e/st .
cp -R NicolasPetton-jtalk-20cd63e/js .
cp -R NicolasPetton-jtalk-20cd63e/css .
cp -R NicolasPetton-jtalk-20cd63e/ide .
rm -rf NicolasPetton-jtalk-20cd63e

JTalk stores source code in three forms: Smalltalk code, Javascript code and smaller Javascript code (“*.deploy.js”).

Jtalk hello world: the Counter example

now, edit app/views/layouts/application.html.erb so that it looks like this:

<!DOCTYPE html>
<html>
<head>
  <title>Jtalk On Rails</title>
  <%= stylesheet_link_tag :all %>
  <%= javascript_include_tag :defaults %>
  <%= csrf_meta_tag %>
  <script src="js/jtalk.js" type="text/javascript"></script>
  <script type="text/javascript"> loadJtalk()</script>
</head>
<body>
<button onclick="smalltalk.Browser._open()">Class browser</button>

<div id="counters"></div>

<script type="text/javascript">
  jQuery(document).ready(function() {'#counters'._asJQuery()._append_(smalltalk.Counter._new())});
</script>
<%= yield %>

</body>
</html>

Here, we included a button to open the code browser, and added a Counter in a div. Oh, I forgot to tell you: Jtalk works seamlessly with JQuery :)
Now, go check it out, and you will seee the counter and be able to increase and decrease the value displayed (yes, that’s a counter).

Editing the code

Click on the “Class browser” button to start the IDE. Select the “Examples” category, the “Counter” class, the “actions” method category, and the “increase method”. You will see in the text box below the source code of the increase method:

increase
count := count + 1.
header contents: [:html | html with: count asString]

Edit that method to increase by steps of 2 instead of 1, and hit “Save”. Now, the counter on your page will increase by steps of 2.
Unfortunately, on the next page refresh, you will lose these changes. That’s why the “Commit category” button is there.
It will take the updated files (here, Examples.st, Examples.js and Examples.deploy.js) and make a PUT request to their original URL.

A PUT, you said? Well, I can work something out with a PUT.

Saving the code

Let’s create a new controller, called Uploader:

rails generate controller uploader jtalk

And edit config/routes.rb as follows:

Jtalkonrails::Application.routes.draw do
  root :to => "home#index"
  if Rails.env == 'development'
    put 'st/:id' => 'uploader#jtalk'
    put 'js/:id' => 'uploader#jtalk'
    put 'js/:id.:deploy' => 'uploader#jtalk'
  end
end

Now the PUT requests are redirected to our controller, but only in the development environment. You do not want to make your JS editable from the browser in a production app. DO NOT WANT!

The only thing left is the controller itself:

class UploaderController < ApplicationController

  def jtalk
    path = Rails.root.join('public')
    if(params[:format] == "js")
      path = path.join("js")
    elsif(params[:format] == "st")
      path = path.join("st")
    end

    if(params[:deploy])
      path = path.join(params[:id]+".deploy."+params[:format])
    else
      path = path.join(params[:id]+"."+params[:format])
    end

    File.open(path, "w") do |f|
      f.write(request.body.read())
    end

    head 200
  end

end

Here, we build the file path from the parameters. I use request.body.read() to get the file content because Rails seems to truncate the beginning of the file.

Profit

Now, go back to the web page, click on “Commit category”, and refresh the page. Your changes were saved! You can enjoy editing your frontend directly from the webpage itself, in the code browser, and more importantly, write your whole frontend in Smalltalk! It’s still missing the workflow “edit-try-debug-edit-continue”, but it already feels just like a “normal” Smalltalk environment. It feels like home :)

Post Scriptum

If you want to add a new category, it’s easy: create a file Mycategory.js and put it in public/js, with this content:

smalltalk.addClass('Myclass', smalltalk.Object, [], 'Mycategory');

and change your initialization from loadJtalk() to loadJtalk(new Array(“Mycategory.js”)). The new category will now appear in the code browser, and clicking on “Commit category” will create the deployment file and Smalltalk source file.

Rails and oauth-plugin part 2: the consumer

In the previous post, I showed how you could build a provider with oauth-plugin and Rails. Now, I will demonstrate how to build a consumer (it’s a lot easier).

I will assume that your provider is already running on localhost:3000. The consumer will run on localhost:4000 (run it with “rails server -p 4000″).

Here we go!

rails new consumer
cd consumer

Put this in your Gemfile:

source 'http://rubygems.org'
gem 'rails', '3.0.7'
gem 'sqlite3'
gem 'devise'
gem "oauth-plugin", ">= 0.4.0.pre1"

And run these commands:

bundle install
rails generate devise:install
rails generate devise User
rake db:migrate
rails generate controller welcome index
rm public/index.html

And here is your routes.rb:

Provider::Application.routes.draw do
devise_for :users
root :to => "welcome#index"
end

Create the consumer

rails generate oauth_consumer user
rake db:migrate

in app/controllers/oauth_consumers_controller.rb, replace:

before_filter :login_required, :only=>:index

by

before_filter :authenticate_user!, :only=>:index

Uncomment the methods for devise (go_back, logged_in? currentuser=, deny_access!) in app/controllers/oauth_consumers_controller.rb.

Add to app/models/user.rb:

 has_one  :test, :class_name=>"TestToken", :dependent=>:destroy

Now go to http://localhost:3000/oauth_clients/ to register your first application with these parameters:

Name:                 Test consumer
Main Application URL: http://localhost:4000/
Callback URL:         http://localhost:4000/oauth_consumers/test/callback

You’re redirected to http://localhost:3000/oauth_clients/1. It shows:

Consumer Key:      CRcIJ15MwSqlDTxsH8MpO3En4wjaOxkqeofLioH4

Consumer Secret:   C7uci8xkyMShCf4SNXWPclKbBo3ml1Zf2W2XWu4W

Request Token URL: http://localhost:3000/oauth/request_token

Access Token URL:  http://localhost:3000/oauth/access_token

Authorize URL:     http://localhost:3000/oauth/authorize

Now, you need to put the key and secret in config/initializers/oauth_consumers.rb:

 OAUTH_CREDENTIALS={
:test =>{
:key => "CRcIJ15MwSqlDTxsH8MpO3En4wjaOxkqeofLioH4",
     :secret => "C7uci8xkyMShCf4SNXWPclKbBo3ml1Zf2W2XWu4W",
     :expose => true
   }
 }

Create app/models/test_token.rb. This model will store the token for your provider. If you want to provide helpful methods, take inspiration from lib/oauth/models/consumers/services/.

class TestToken < ConsumerToken
TEST_SETTINGS={
:site => "http://localhost:3000",
:request_token_path => "/oauth/request_token",
:access_token_path => "/oauth/access_token",
:authorize_path => "/oauth/authorize"
}

def self.consumer(options={})
@consumer ||= OAuth::Consumer.new(credentials[:key], credentials[:secret], TEST_SETTINGS.merge(options))
end

end

You should now be able to use the URLs “/oauth_consumers/test/client/”, “/oauth_consumers/test/callback”, “/oauth_consumers/test/callback2″,” /oauth_consumers/test/edit”,
and “/oauth_consumers/test”.

Modify the welcome controller t get the provider data:

 class WelcomeController < ApplicationController
 def index
 # cf http://oauth.rubyforge.org/rdoc/classes/OAuth/AccessToken.html
 @consumer_tokens=TestToken.all :conditions=>{:user_id=>current_user.id}
 @token = @consumer_tokens.first.client
 logger.info "private data: "+@token.get("/data/index").body
 end

end

To connect a user to an external service link or redirect them to:

/oauth_consumers/[SERVICE_NAME]

Where SERVICE_NAME is the name you set in the OAUTH_CREDENTIALS hash. This will request the request token and redirect the user to the services authorization screen. When the user accepts the get redirected back to:

/oauth_consumers/[SERVICE_NAME]/callback

That’s it

This tutorial is really short, and could be explained a bit more, but I’ll leave that for another post. You have enough to start tinkering with OAuth. Have fun!

Rails and oauth-plugin part 1: the provider

These days, I have been playing a lot with Oauth and its RoR implementation, oauth-plugin. Its documentation is a bit short, so here is a tutorial to show how to use it, both in provider and consumer mode. And we will even make them communicate with each other.

We will now build an Oauth provider using oauth-plugin for authorization and Devise for authentication. And we will add a controller protected by Oauth.

Starting up

A few instructions to create the application. You won’t need an explanation for this:

rails new provider
cd provider

Put this in your Gemfile:

source 'http://rubygems.org'
gem 'rails', '3.0.7'
gem 'sqlite3'
gem 'devise'
gem "oauth-plugin", ">= 0.4.0.pre1"

And a few more commands:

bundle install
rails generate devise:install
rails generate devise User
rake db:migrate
rails generate controller welcome index
rm public/index.html

And don’t forget ‘root :to => “welcome#index”‘ in config/routes.rb.

Create the provider

rails generate oauth_provider oauth

rake db:migrate

You could put something else than “oauth” as parameter, but for the moment, the generator has some bugs (it always generate the class OauthController, but with a different name). I’ll check more recent versions of the code.

Now, modify config/application.rb and add:

require 'oauth/rack/oauth_filter'
config.middleware.use OAuth::Rack::OAuthFilter

Put in app/models/user.rb:


has_many :client_applications

has_many :tokens, :class_name=>"OauthToken",:order=>"authorized_at desc",:include=>[:client_application]

Put in app/controllers/oauth_controller.rb:


alias :logged_in? :user_signed_in?

alias :login_required :authenticate_user!

and uncomment authenticate_user.

Put in app/controllers/oauth_clients_controller.rb:

alias :login_required :authenticate_user!

And now some data

Create a new controller:


rails generate controller data index

And now, edit your controller:

class DataController < ApplicationController
  before_filter :oauth_required

  def index
    @data = { "coincoin" => "o< o<" }

    respond_to do |format|
      format.json { render :json => @data }
    end

  end
end

UPDATE

I discovered a few bugs in this tutorial, so here are the fixes.

oauth-plugin needs the function current_user=, so add this to your ApplicationController:

def current_user=(user)
  current_user = user
end

Next, to handle revocation, you need to add this to config/routes.rb:

post 'oauth/revoke'

And at last, you need to fix the rack filter. The current code doesn’t verify the token validity, and lets revoked tokens access your data.
You have to modify lib/oauth/rack/oauth_filter.rb in the oauth-plugin gem folder.
Replace the line 46:

oauth_token = client_application.tokens.first(:conditions=>{:token => request_proxy.token})

by

oauth_token = ClientApplication.find_token(request_proxy.token)

And that’s it!

You now have a working provider. OauthController handles all the communication with the consumers. OauthClientsController manages the registration of new consumers. They both have customizable views: oauth for the authorization part (for users) and oauth clients for the consumers. And you just need the oauth_required filter to manage access to your data.

And now, you can go to /users/sign_up, then /users/sign_in, then /oauth_clients to register a new client application. You just need to give a name for your application, your URL, and a callback URL.

In the next post, we will build a consumer, and this consumer will access the provider’s data.