Lock table is out of available locker entries

I managed to shaft my Subversion repository today. No idea how it happened, and fixing it was a bit problematic. The error was: svn: bdb: Lock table is out of available locker entries If I accessed the repos directly through the web (Apache), it just said “couldn’t open the requested svn filesystem”. WebSVN gave the above error, as did using svnlook on the repository itself. The problem is that the Berkeley DB database* underlying the repository has run out of locks. To fix this, there’s the short-term fix and the long-term fix. You should do both: you have to do the short-term fix first because until you do nothing else works, but then you should really do the long term fix as well; it’ll stop this ever recurring again.

The short term fix

What we have to do is increase the number of lockers in the database. There are two steps to this. First, edit /path/to/your/repos/db/DB_CONFIG. Buried in there there will be some of the following lines:

set_lk_max_locks   2000
set_lk_max_lockers 2000
set_lk_max_objects 2000
set_lg_bsize     262144
set_lg_max      1048576

I’m not sure exactly which to change, so I changed lots of them, as follows:

set_lk_max_locks   3000
set_lk_max_lockers 3000
set_lk_max_objects 3000
set_lg_bsize    2097152
set_lg_max      8388608

However, that alone won’t work. Berkeley DB seems to cache the settings. To fix this, you need to remove the cached environment. In /path/to/your/repos/db/ there will be files called __db.001, __db.002, etc. You need to remove these files. I strongly recommend that you move them somewhere, not delete them. Once you’ve removed them, try re-accessing the repository the way you normally do and it should work. If it didn’t work, move the files back before you try anything. Now do the long-term fix, below.

The long-term fix

The real problem here is that Berkeley DB isn’t very good. The svn people advise you to not use it now in favour of their own format, fsfs. I’m sure that BDB is a great and powerful program, but it does seem to get lots of locking errors and so forth. So, you should convert your repository to fsfs format. Do this like so:

mv /path/to/repos /path/to/repos.old
svnadmin create -fs-type fsfs /path/to/repos
svnadmin dump /path/to/repos.old | svnadmin load /path/to/repos

That renames your existing repository to repos.old and then creates a new fsfs repository where the old one was. Finally, it transfers all your data from the old one to the new one. You can delete the old one once you’ve done this, but it wouldn’t hurt to keep it lying around in case something goes wrong. Once you’ve done this, everything should continue working, and you’ll never get locking errors again.

I'm currently available for hire, to help you plan, architect, and build new systems, and for technical writing and articles. You can take a look at some projects I've worked on and some of my writing. If you'd like to talk about your upcoming project, do get in touch.

More in the discussion (powered by webmentions)

  • (no mentions, yet.)