Jan/105
Robust DBI Transaction and Connection Handling
Edit: It turns out there is already a module on CPAN that does exactly what I talked about here. Its called DBIx::Connector. I haven’t tried it yet, but it looks like the guy that wrote it designed it with input from the DBIC guys.
I’m a big fan of DBIx::Class. Among DBIC’s many great features is its superb transaction and connection handling via DBIx::Class::Storage::DBI. When I’m using raw DBI I feel like I’m missing these core components. I should always be able to expect robust transaction and connection handling.
I started playing around with creating a new CPAN module (something like DBIx::Robust, DBIx::Transaction, etc…) but each time I dove in to it I kept coming to the conclusion that DBIC’s DBI storage drivers provide everything I need and I would just be shooting myself in the foot by either porting DBIx::Class::Storage::DBI to a DBIC independent API or by writing it from scratch.
Before I go any further, let me show you how attractive and awesome DBIC’s storage layer is:
- Transactions may be nested, even without savepoints.
- Transaction savepoints can automatically be tracked allowing for reliable and incremental rollbacks.
- The appropriate DateTime::Format class is automatically determined based on the type of database.
- All database calls efficiently pass through a layer that detects stale connections that will attempt to re-connect to a database.
“There is no charge for awesomeness, or attractiveness.”
The DBIx::Class::Storage::DBI pod illustrates all this in a round-about way: “If you set AutoCommit => 0 in your connect info, then you are always in an assumed transaction between commits, and you’re telling us you’d like to manage that manually. A lot of the magic protections offered by this module will go away. We can’t protect you from exceptions due to database disconnects because we don’t know anything about how to restart your transactions. You’re on your own for handling all sorts of exceptional cases if you choose the AutoCommit => 0 path, just as you would be with raw DBI.”
This stuff is really powerful, and we should never have to work with databases in Perl without it. But, currently, it is only meant to work under DBIC, which is a shame. I’m betting that DBIx::Class::Storage::DBI could be used directly, bypassing DBIC completely. There is some DBIC stuff layered in there, but that should be ignorable.
What do the rest of you think? Has anyone considered moving this logic out in to a generic distribution that isn’t DBIC specific? Is there merit in what I’m talking about? Does anyone use DBIx::Class::Storage::DBI directly, bypassing DBIC?
Jul/090
Scientific Troubleshooting
Over the years I’ve naturally applied scientific principles to the way I program. There are two main principles that I think most good programmers ought to know.
Occam’s Razor (Wikipedia)
The idea behind Occam’s Razor, when applied to programming, is that when you are presented with a problem you should pick the simplest solution first and go try and prove it. We naturally formulate possible reasons (hypothesis) in our mind as soon as we are presented with a problem. Most junior programmers will pick either the first, or the easiest, hypothesus and go forward blindly. Neither of these is the simplest.
The easiest one is not a good choice because easy has a different meaning for every programmer in every different situation. Easy could mean that you are willing to sacrifice hours injecting print statements all over 20,000 lines of code. Easy could mean that you don’t understand the underlying system, so, rather than dig in a little and understand how things are working behind the scenes, you opt to stay safe and kick the tires hoping that something will present itself to you.
Picking the first hypothesis is an even worse choice because you’re not even attempting to prioritize the steps you are going to take to find a solution for your problem. Instead you are just blindly jumping in and will likely waste hours determining that your first hypothesis was not correct.
The bottom line is when you are presented with a problem it is important to consider several possible reasons for the problem, choose the simplest reason, and then go forward. More often than not the simplest solution is also the fastest one to confirm, which allows you to get through your initial list of hypothesis fairly quickly. At this point you will have identified the problem, or will have ruled out a bunch of stuff that will allow you to better test your larger hypothesis.
Scientific Method (Wikipedia)
The scientific method, when applied to programming, is a set of steps to take when troubleshooting an issue. In many cases following the scientific method is overkill, as abiding by Occam’s Razor is usually more than enough. For larger issue it helps to apply the scientific method which helps you structure your course of action in to various steps and can give you a visual map of all the steps you need to take to diagnose, and fix, your issue. The steps are:
- Purpose – State your issue.
- Research – Find out about the topic, if necessary.
- Hypothesis – Predict a reason for the issue.
- Experiment – Develop a procedure to test the Hypothesis.
- Analysis – Record the results of the experiment.
- Conclusion – Compare the hypothesis to the experiment’s conclusion.
It is important to come up with several hypothesis, ranging from very simple to very complex, and repeat steps 3 through 6 for each one. Also, remember Occam’s Razor when picking which Hypothesis to test first.
I find it works best for me when I get a pen and a pad of paper. Steps 1 and 2 will be documented on the first piece of paper. Then each hypothesis, steps 3 through 6, will get their own sheet of paper. Sometimes I can get everything to fit on one sheet, but I usually have to overflow on to a second piece of paper anyways, so its good to plan for that up front.
These techniques apply to any programming language such as Perl, JavaScript, HTML, CSS, SQL, etc, and have saved me, and can save you, countless hours and frustration.
Iron Man
Planet Perl Iron Man Posts
- Jerome Quelin: perl's state in mandriva cooker
- Kent Fredric (kentnl): Git Internals: An Executive Summary in 30 Lines of Perl, for smart newbies.
- Jared: Does Your Perl Suck if You Don’t Use Moose?
- Tokuhiro Matsuno: cpan-outdated が local::lib に対応してた
- Steven Haryanto: Startup overhead still matters
- Andrew Shitov (ash): Габор Сабо. Скаляры в Perl 6
- Nuno Carvalho (smash): fork'ing your world
- Andrew Shitov (ash): Заготовка
- David E. Wheeler (Theory): Introducing MyTAP
- Thomas Fahle: Lesetipp: 15. Ausgabe des Perl-Magazins $foo erschienen