lixo.org

Ever had nasty problems coming from multiple development environments set up differently on your team, with developers accidentally creating files with bad encodings? Are your users complaining about all that mojibake showing up on your internationalised UI?

Well, I had… plenty of times. So I created a tiny little tool to help fight that. Encoding Detector is an aptly named tool (if I do say so myself) that recursively detects the encoding of files in a project directory errors out if anything seems fishy.

I’ve only ever used it with Ant, but it should be a breeze to set up and install on any project. All you need to do is call the main script against a directory, like:

$ python encoding-detector.py src

You’ll need a somewhat recent Python installed — anything that came out in the last 5 years should be OK — and that’s about it. Multiple directories can be passed as arguments, and fixing the errors is sometimes as easy as adding a few UTF-8 characters to boost the detector’s confidence up a bit.

This hack would definitely not be possible without Mark Pilgrim‘s amazing chardet library. The code, as always, is on GitHub, and you can also grab a neat little tarball here.

It takes about 5 minutes to set up on your project, and you can thank me later

As many reading this might know, I’ve moved from London back to São Paulo about a month ago. The timing worked out wonderfully: right before a bunch of really interesting conferences happening over here. Among them, I’m speaking at:

Encontro Ágil, 10-11 October: a quick talk about the spirit of continuous integration and how important it is to stick to it even though it’s easy to get sidetracked choosing between the tools available. I’ll explain some of the pitfalls, patterns and anti-patterns, and hope to please the Demo Gods by showing what a good continuous build, test, integration and deployment cycle looks like in practice.

Rails Summit 13-14 October: another quick talk about the use of Ruby at ThoughtWorks and some of the lessons learned in the 40+ projects we’ve done so far. I’ll try not to make it a total rip-off of Martin’s Ruby at ThoughtWorks work for QCon earlier this year, but I’m making no promises.

More to come, I’m sure. It’s good to be home!

A side note: would you, kind reader, get annoyed if I start mixing Brazilian Portuguese and English in future posts? Should I set up a separate blog, category or whatever so the non-lusophones can skip the stuff written in what would make Camões spin in his grave? Any suggestions as to how to do that effectively in WordPress?

In most projects I have been on as a consultant, getting the right environments together was an incredibly consuming task on several people with too much to worry about already: project management, senior techies and the ones responsible for keeping the legacy environments running.

Recently, the idea of getting an example minimal application deployed as soon as possible during the project lifecycle has been banded about a bit as one of the things that could’ve made my last project run smoother (as always, hindsight is always 20-20). Then, one of my colleagues (either Graham Brooks or Sam Newman, but I could be mistaken) brought up a simple and perfect analogy for it, found in The Pragmatic Programmer: a tracer bullet.

From Wikipedia:

Tracer ammunition (tracers) are special bullets that are modified to accept a small pyrotechnic charge in their base. Ignited upon firing, the composition burns very brightly, making the projectile visible to the naked eye. This enables the shooter to follow the bullet trajectory relative to the target in order to make corrections to his aim.

Tracer bullets are used during the daytime as well, but their importance is heightened severely at night: when shooting in the dark, being able to see the trajectory is crucial, since other clues — such as surrounding objects being hit — are harder to recognize and do not provide quick enough feedback for the shooter to be able to adjust his aim in a timely fashion.

Since most software projects are, in a way or another, shooting in the dark, it makes sense to create a special type of application deployment that increases the visibility of the deployments coming after it.

This application does not need to have any features, besides painting an accurate picture of what the real application (coming up in the next few deployments, hopefully) will behave like. A few interesting things to analyze:

• System environment: is the PATH set properly? Are the versions of compilers, interpreters and runtimes correct? Is there enough memory and storage space?
• Library dependencies: are all the dynamically linked libraries available and in the correct versions? What’s the overhead of a particular choice of framework or data mapping strategy in memory consumption and performance?
• Interactions with other legacy systems: are they available in the correct version? Is the application authorized to talk to them? Can they be contacted reliably and at the performance levels desired, or is there a network issue that needs to be solved?
• Performance: how many users of the application (once it’s deployed) can we tolerate? What are the expected response times? Under load, what are the bottlenecks?

These questions do not necessarily have to be answered by a monolithic tracer bullet application. In my view, they’re more like deployment spikes: once they answer a question and the team learns from them, they can be thrown away or reused, but there’s no obligation to build on a previous one if doing so adds noise to the measurements.

This works for me: testing is about sustainability. If your tests don’t let you be sustainable (for whatever value of “sustainable” is
important to you) then you’re probably doing something wrong.

The interesting thing here is that you can define sustainability as “if we don’t have this live tomorrow morning, the company will fold” and then proceed with the most appropriate testing strategy – something that gets out of the way and gives you the fastest possible feedback, like a smoke test not much deeper than a ping to the server or a wget piping to diff.

If you define “sustainable” to be “this codebase needs to constantly adapt to new features and output formats over the next 5 years”, the testing strategy changes accordingly, and it is up to a team of competent professionals to choose at what level to operate. In this case, most would choose to use tools like Behaviour Driven Development/Design and Domain Driven Design to explore as much of the problem space as possible during development and seek shared understanding through automated and repeatable processes, that can be maintained along with the codebase and ensure the features are always easy to understand and modify, should the functionality require changes (and it will).

Git commits: 1688
Xbox 360s bought: 1
Projects created or started: 14
Guitar Hero levels completed with 5 stars: 98 (on medium)
Burglaries: 2
Full-time projects: 3
kiva.org loans made: 7
Languages learned: 2 (Erlang, LOLcode)
Presentations given: 3
Family Guy episodes watched: 116
Presentations attended: 12
Interviews given: 2 (akitaonrails.com and MTV)
Tweets: 979
Flats: 1 bought, 1 lived in
Arduinos: 4 bought, 2 destroyed
Non-fatal incidents with a soldering iron: 3
Flight time: 28.5 hours
Countries visited: 4 (Brazil, France, Ireland, UK)
Google Reader items read: 127524
Pints of London Pride per week, average: 4.33
Books: 18 bought, 7 read
Photos taken: 29355
Semi-naked photo shoots published: 1
Copies of said publication sold: 500k+
Blog posts written: 16
Burritos eaten: 47
Attempts at purchasing an iPhone: 9 (0 of which successful)
Emails written: 1045 (2.8 per day in average)
Times I heard George say “shiiiiiiit” per day, average: 2.19