Tigraine

Daniel Hoelbling-Inzko talks about programming

Mongoid i18n and fallback locales

I may have hinted at it once or twice that I really love how Rails makes Internationalization (I18n) really simple (in stark contrast to what I had to do back then in .NET). But one thing Rails I18n will not do for you is translating the models in your database, which is to me one of the major problems I often face at work when doing I18n. (Of course there are gems that alleviate that pain)

So today I was sitting down to write a system design for a new application at work that has to be localized in a lot of different languages while still being fast. After thinking a bit about the problem I decided that doing so in ActiveRecord would be cumbersome to say the least, and I decided for a Document-Database approach. Since I know MongoDB best I went out and looked at the Mongoid documentation on how to best integrate transparent localization into the system (fully expecting that I'd have to implement this myself).

Well, Mongoid blew my mind:

class Post
  include Mongoid::Document

  field :title, localized: true
end

That's it, Mongoid will now transparently store the title field in a hash and return the correct value depending on I18n.locale. We are talking literally zero cost here. Mongoid even accounts for the reality of translations not always being present so you might want to fallback to another locale instead of not presenting anything. Wow..

So here is how to set this up with fallbacks inside your config/application.rb:

config.i18n.fallbacks = true
config.i18n.default_locale = :en

That's all, and here is how it works:

post = Post.new(:title => 'Hello World')

puts post.title # => "Hello World"
I18n.locale = :de
puts post.title # => "Hello World"

post.title = "Hallo Welt"
puts post.title # => "Hallo Welt"

I18n.locale = :en
puts post.title # => "Hello World"

Unfortunately this at the moment only works for regular fields, not for relations so I still have to do some work in my current project - but even the field i18n saves me a lot of headache.

Multiple fallbacks: Something that is not really covered in the documentation but becomes apparent when you look at the tests is that you can define a callback chain per locale. Meaning you can tell Mongoid to do go through a list of locales while searching for an existing value.

::I18n.fallbacks[:de] = [ :de, :en, :es ]

Mongoid will now first look for a german value, then fall back to the english one, and if that's not present either it will try the spanish one. Very handy especially if you run the locale schema down to country codes like 'de-AT', 'de-CH'. So for example this is the fallback chain for switzerland:

::I18n.fallbacks['de-CH'] = ['de-DE', 'de', 'en']
::I18n.fallbacks['fr-CH'] = ['fr-FR', 'fr', 'en']
::I18n.fallbacks['it-CH'] = ['it-IT', 'it', 'en']

Yes, Switzerland is a mess - but at least it gives you a good example for crazy fallbacks.

Filed under mongoid, ruby

New blog engine, this time my own

It's been some time since I did write a blog post, and I must say it was not for a lack of ideas. It was mostly the prospect of having to write these posts in HTML into my Wordpress interface that led me to abandon the idea.

And I think that's the main problem: I believe HTML is a bad format for writing posts. And no amount of WYSIWYG will change that, for all the subtle codehighlights I like in my posts the Wordpress WYSIWYG is just not the least bit useful. And don't get me started on posting source.

There are basically 2 solutions I had with my Wordpress installation: Either use a clientside syntax-highlighter or rely on Gist to post code in a human-readable fashion that's also easy on the eyes.

And I hated that.. I hated doing the back and forth between Gist and the site, and fixing errors in the code was always a pain. And as much as I like the syntax-highlighter script, it slows the page down a lot. Especially the front-page, even when the user may not be looking at a post with code at the moment code on the page will still be parsed.

A good solution to this problem seemed to be Jekyll, where you write your posts in simple markdown or textile and code gets syntax highlighted automatically on the server. I tried that some years ago, but for my style of writing it turned out to be even worse than Wordpress.

Why? Simple: I didn't start from scratch but rather imported my whole Blog into Jekyll when I started the experiment. So I was running Jekyll with syntax highlighting on on about 300 posts all the time. And the compilation did run for about a minute every damn time! To add insult to injury: at the time I didn't have a rsync capable hosting setup so I had to FTP the whole site.

So a few weeks ago I started to tinker around with Ember.js, and naturally I found myself writing a sample blog application. And that's when it hit me: writing a blog is f***ing easy. There is a reason any web frameworks worth it's salt is demoed using a blog engine.

And it turns out, what few features I want from my blog engine are usually all covered by these very simple demo apps :). So I ditched all the Ember.js experiments and started to write my own blog engine that had to do the following things:

  • Look exactly as my old blog
  • Accept Markdown input
  • Provide syntax highlighting
  • Let me schedule posts to appear at a later date
  • Preserve permalinks

And that's it - almost laughable how simple my requirements actually are. I decided to ditch some more complexity I felt no longer required (since RSS is dead) and built a really simple rails application ontop of the awesome libraries rouge and redcarpet. So my blog now accepts GitHub Flavored Markdown - all with less than 5 lines of code.

And the results are really awesome:

puts "I <3 Ruby"

So I sat down and built the blog in probably half a day. Maybe a bit more since I wanted to have some shiny like: nice tagging with autocompletion, real time post preview while typing and drag & drop file upload for images.

I'll post the code once I cleaned up the deployment mess I created (no secure database.yml management, secret.rb in git etc) but once done I'll post it here anyways.

Next up for me is trying to figure out a good way to make the blog theme-able. I'd love to experiment a bit with Rails engines and how this can be leveraged to separate themes from applications. And now that I've got my own little Rails playground once again it's the perfect opportunity to do so :).

Oh, and I need to leverage caching. After a cursory exploration today it seems caching would turn this blog into a statically generated static site. Only problem seems to be the cache invalidation, but I am sure I can devise some scheme to get around that that (short of doing 5 rm -rf calls).

Filed under rails, ruby, blog, site-news

Amazon S3 uploads fail in numerous ways for no apparent reason

Today I spent almost 30 minutes trying to debug a problem where one of our Servers was not correctly synchronizing files to Amazon S3.

First thing I tried was manually doing a s3cmd put of the files in question, and was immediately greeted by numerous connection reset by peer error messages. s3cmd is so smart, it even automatically retries for 5 times while throttling the upload bandwith - still to no avail. When down to 5 kb/s upload speed I started getting broken pipe error messages.

Well, obviously our server has more than 5 kb/s uplink so I suspected someone broke something on the network level, but everything else was working fine.

The final clue then came when I was running `s3cmd ls` on my bucket where I finally got a meaningful error message: the difference between the request time and the current time is too large. Huh? Yes! Doing a date on the server revealed that the server was almost 20 minutes behind the current time, thus the SSL connection to S3 could not be established.

Turns out we had not enabled ntp and so the server clock kept drifting for a couple of months. The solution was straightforward:

sudo ntpdate ntp.ubuntu.com
sudo apt-get install ntp

The first does an immediate update (only installing ntp will take some time to sync the clock back up, so doing a manual update fixes your problem immediately), and then it installs ntp to prevent this from happening in the future.

Filed under tools

Installing Eclipse on Mac OSX

Most Mac applications come bundled as a .app package that you simply have to drag&drop into your Applications for them to show up in Launchpad and Alfred. Or, if an application is a bit more complex you usually get a .dmg file that then installs like any normal MSI package you are used to from windows.

Not Eclipse for OSX. The Eclipse project maintainers do provide a OSX binary, but it doesn't look like most OSX .app packages in that the extracted binaries are a folder that then contains a .app.

If you only copy over the .app file you'll not be able to start Eclipse as it's just a starter script and the real Eclipse is living in the extracted Zip. I used to just start eclipse wherever I unpacked it, but apparently the right way to install it is to simply copy the whole extracted eclipse folder over to your Applications. Even though the .app is not in /Applications directly OSX picks it up and it shows up in Launchpad.

Filed under programmierung, java

When to use raw() and when to use .html_safe

What rails does very nicely is protect you from XSS attacks through aggressively HTML encoding anything you write between <%= %>. But there is one caveat: At times you may really want to render HTML from a string. So you need to tell rails not to escape your HTML in that case.

There are two methods of telling rails that a string is safe and should not be escaped: raw and .htmlsafe

And both do the same. They mark the string as safe (through the use of the html safe buffer) and rails will not encode it any more. The main difference between the two: nil.

If you are doing things like: "<img src='#{..}' />".html_safe .html_safe is totally fine as the string will never be nil, but if you are dealing with strings that may be nil .html_safe will break since there is no .html_safe method on the nil object. (For example if you are loading something from a config value or the database) In that case using raw(...) will just ignore the string instead of raising an exception.

As always with these things: raw and html_safe make it very easy to introduce XSS attack vectors into your application. So use them wisely and only on strings you are sure to be safe.

Filed under programmierung, rails, ruby

How to expire all active Sessions in Rails 3

I know this sounds like a very simple task to do, but since I just spent half an hour reading up on how Sessions in Rails work I decided it's time to put this up so I can Google it again next time :)

First off, you need to know how you are storing sessions in your application. Rails supports 3 types of session storage: Stored on Disk, in the Database or through Cookies in the client. You can check which one you are using in /config/initializers/session_store.rb.

Cookie based session storage

Rails 3 defaults to storing the session in the client using a session cookie. This means that the user_id along with all the data you put into the session hash is serialized into the cookie and sent to the client. It's also not encrypted, only BASE64 encoded so if you are storing anything sensitive in there you are doing it wrong.

But for simple things like the current user_id the cookie based session store is just fine and also a lot faster than the alternatives.

Expiring the cookie though is a bit more involved since you can't reach out to all clients and delete their cookies at once. But, and that's the important part for what I was doing: This cookie is signed with a SHA-512 digest using a secret key that is only present on the server. So the cookie cannot be tampered with on the client, and this is also your avenue of attack when trying to expire all cookies:

Simply change the secret that is used to sign the cookies. All previous cookies are invalidated as their digest is no longer valid.

Doing so is simple, first generate a new secret using rake:

$ rake secret 10dfec4781b682762a731a5e88af78521fc3e0f...

Then copy/paste that new secret into your config/initializers/secret_token.rb:

MyApp::Application.config.secret_token = '10dfec4781b682762a731a5e88...'

Once done deploy the application and all existing sessions are invalid at once.

Database backed session storage

If you are using the Database to store the session it's rather trivial to expire all existing sessions using rake:

rake db:sessions:clear

File based session storage

Like with the database simply run the following rake command:

rake tmp:sessions:clear

Hope this helps..

Filed under programmierung, rails, ruby

Optional locals in Rails partial templates

In Rails it is very advisable to not use any instance variables inside your partials if you want to re-use that partial in a different context.

It is much better to simply leverage locals that you pass into your render like this:

That way you can simply use posts inside the template without having to rely on @posts coming from the controller (and it's also not dependant on controller code any more but rather only on data inside the view that's calling it.. very handy).

But, as always there are times when you have additional locals that are optional to only some contexts. In my case I had Kaminari paginate the list I was passing to the partial, but in some other views the list was not be paginated. Turns out you can't do the usual rails style if foo here because the variable is simply not defined and Ruby will throw an error (unlike with @variables in Rails).

That's where the defined? method comes in very handy (as suggested by Thorsten Ball):

If you simply want to assign a default value to a local you can use the ||= operator:

This enables me to use the partial in several ways without having to re-introduce the locals at every corner:

Filed under programmierung, rails, ruby

Explicit path helpers with STI in Rails

I am currently working on an application that makes use of Single-Table-Inheritance (STI) for some of it's models.

STI in Rails is quite simple, you simply subclass a model and add the type column to your table like this:

And the accompanying migration:

Now that that's out of the way, Rails url helpers will automagically use the right route depending on the class you feed to it's helpers. Here's an example:

You get the idea, rails helpers are smart about this. Only problem: How about the cases where you need an explicit helper like edit_person_path?

This is where it get's tricky:

Using the explicit helper rails will not do any STI magic and simply return the path you asked for. What I really needed was for a helper that points to /persons/:id/edit in case it's a Person record and to /students/:id/edit for students. After some digging (Rails STI isn't really the hottest topic on Google it seems) I found the polymorphic_path method that does exactly this:

Filed under rails, rails-2, ruby, ruby-2

Removing delete and destroy in Rails models

Today I had an interesting feature request to implement: Get rid of all deletions and replace them with a deleted flag in the DB.

It's the usual story: Nobody really does deletes but rather everything is put into the DB in case it's needed at some later point. Unfortunately I didn't know about this particular feature until after I had finished most of the coding for the Rails application so going through the code and removing all destroy code from controllers was pretty much out of the question.

Instead I remembered reading about Modules in Ruby and how they can bolt on functionality to Classes.

Turns out it's totally trivial to remove deletion from Rails Models with 10 odd lines of code in a completely transparent an unobtrusive way:

module NotDeleteable
  def destroy
    unless self.respond_to? :deleted
      raise MissingMigrationException
    end

self.update_attribute :deleted, true end

def delete self.destroy end

def self.included(base) base.class_eval do default_scope where( :deleted => false ) end end end

class MissingMigrationException < Exception def message "Model is lacking the deleted boolean field for NotDeleteable to work" end end

This will override the default destroy/delete method provided by ActiveRecord::Base and also install a default_scope into the Model class so Rails will by default append WHERE deleted = false to all SQL queries made through the ActiveRecord Query Interface

You use this by simply including this module inside your Model class:

class User < ActiveRecord::Base
  include NotDeleteable
end
Did I mention that I really like Ruby?

Word of warning: I have no clue if I am breaking :dependant => :destroy on ActiveRecord relations in any way, but I suspect it should still be alright.

Update: The original code had an issue where you could not mark a record that is invalid as deleted. This was due to the fact that I was using save instead of update_attribute.

Filed under programmierung, rails, ruby

A word of advice on Bundler and Gems from git

At times (especially after Rails version upgrades) some Gems don't work out of the box so I usually spend some time on GitHub looking through the network graph for a fork that fixes the issue.

This works great and you usually end up with a well working plugin to continue working. Bundler supports this by loading the gem from the git source:

gem 'rails', :git => 'https://github.com/rails/rails.git'

Only problem with this approach: Forks are usually a moving target in regards to their longevity. People tend to delete them once the pull request gets merged, or simply forget about them.

Neither is desirable and today it bit me for the first time, I had referenced a fork that got deleted but contained a important fix for my project on a gem that was not yet 3.1 ready. Fortunately the project has moved along and the master already included the fix (and didn't introduce new issues). But it sure taught that I should always keep my own fork of code I decide to depend upon.

Filed under uncategorized

Projects

dynamic css for .NET