The Problems with Twitter's Automatic URL Shortening

At the beginning of 2010, I decided to start writing up my thoughts on all of the first-run movies that I see in the theater. It's debatable about whether those reviews are any good, but I know that at least some people read them. All of my reviews from the last year and a half are available at http://www.viewity.com/.

Last Thursday, I saw (but did not particularly enjoy) J.J. Abrams' new movie Super 8, and last night I finally got around to writing my review of it, which I posted at http://www.viewity.com/reviews/super-8.html. I use Squarespace to host the reviews, and one of the services it provides is the ability to define a shorter URL that can be used to reference the content. I took advantage of this and created the path "/Super8" instead of "/reviews/super-8.html". Squarespace also offers support for using multiple domains with the same account, and I have "vwty.us" in addition to "viewity.com". What this ultimately means is that going to "http://vwty.us/Super8" will take you to "http://www.viewity.com/reviews/super-8.html".

Whenever I post a new review, one of the ways I let people know about it is by Twitter. The whole reason that I offer the shorter version of the URL is that Twitter limits posts to a maximum of 140 characters, and at 21 characters, the short version of the URL is less than half the size of the 43-character long form. This gives me more space to say something about the movie in addition to merely providing the link, and I try to give at least a hint about whether I liked it. For Super 8, the tweet that I composed was:

Super 8 is super underwhelming. http://vwty.us/Super8

However, what actually got tweeted was:

Super 8 is super underwhelming. http://t.co/TZ43SmY

I will grant you that what Twitter actually made available on my behalf is a whopping two characters shorter. However, it is also much worse than what I had originally written, for many reasons.

First, it's completely unnecessary. As I mentioned before, Twitter places restrictions on the length of your tweets, but I wasn't anywhere near that. What I originally wrote was 89 characters, which means that I could have written up to 51 more characters before running out of space. I could have even used the original 43-character URL if I had wanted to and still had plenty of space left.

Second, Twitter's change dramatically obscures the URL. From the URL that I provided, you can tell that it goes to the vwty.us domain (which is a brand that I control and want to be associated with), and the "/Super8" path gives you a pretty good idea what it might be about. On the other hand, with what Twitter actually provided, you can see that it goes to the "t.co" domain (which is known to be a redirect farm so you have no idea where the content actually resides), and the path "/TZ43SmY" tells you nothing about the content. The original URL is very useful. The shortened version is not.

Another significant problem is that the new URL shortener can have a dramatic impact on the availability of your content. Twitter has such a bad reputation in this area that their "fail whale" page is a well known Internet meme. Because a click on the shortened URL must go through Twitter's servers before sending you to the ultimate destination, if Twitter is having a problem then it can make your content unavailable. As if by fate, when I clicked on the t.co link earlier this morning, I got exactly that failure page telling me that Twitter was over capacity. Nice. Even if it had worked, it still requires an extra HTTP request and more data over the wire, and an unnecessary delay in getting to the actual content.

The requirement to go through Twitter's service creates even more ways that the content could become unavailable. It's likely that tweets will outlive Twitter itself. They're being archived in the Library of Congress (in addition to a number of other sites), and although future generations probably don't care how I feel about a movie, there could be long-term value in tweets, and links contained in them. If Twitter goes out of business or is otherwise shut down, then their links won't work anymore even if the content they referenced is still available. Also, it's worth pointing out that the ".co" TLD is controlled by the government of Columbia, and that government can shut down such URLs at any time. The government of Lybia has done this for ".ly" domains, so it's certainly not beyond the realm of possibility.

Twitter's reason for providing this service is that it can "better protect users from malicious sites that engage in spreading malware, phishing attacks, and other harmful activity". While this sounds noble, it is also completely ineffective against everyone except the most extreme idiots. They've already stated that they won't shorten URLs that were already shortened using other services like bit.ly, so there's nothing to prevent people doing suspicious things from using one of them for their posts. Further, there's nothing to prevent me from serving up different content from my server when I can see that the request is coming from Twitter's malware detection service versus some other content, so I could still serve up bad stuff to people following the links. On the other hand, the fact that they are trying to verify that content is safe introduces a very real possibility for false positives. My site could have completely legitimate and safe content, but if Twitter thinks that it's bad for some reason then that may significant inhibit the likelihood of people to go there. Given the unacceptably high percentage of false positives I see from other services like this (e.g., Google mail's spam detection frequently flags things that aren't spam), this is far from an impossibility.

Finally, in the ultimate act of inanity, Twitter's URL shortener can actually produce URLs that are longer than the original URL. For example, when I entered a URL of "http://t.co", Twitter "shortened" it to be "http://t.co/IzZPmi2".

I realize that Twitter will show an expanded version of the URL in its web interface, but that doesn't work for alternate clients. For example, when I use Seesmic on my Android phone, I get the t.co version. And even if I'm using a client that automatically expands that URL, it will only work if the shortening service is available.

Great job, Twitter. This "feature" that I can't disable has made my links less available, less recognizable, and more likely to be flagged as malicious content. I don't need any more hurdles to have to get by for people to read the useless drivel that I write.


Comparing Java LDAP SDK Performance

At UnboundID, we take performance seriously and are always trying to improve. This applies just as much for our client-side SDK as for our server products, since a server isn't very useful without client applications to take advantage of it. There are a number of tools that can be used to measure various aspects of directory server performance, but it's not as simple to measure the performance of client-side libraries.

To help address this problem, I've written a simple tool that can be used to perform a number of different kinds of LDAP operations using various Java-based LDAP SDKs. It's not particularly elaborate and there's only a command-line interface, but it provides a range of options, including which SDK to use, the type of operation to test, the number of concurrent threads to use, the length of time to run the test (and to warm-up before beginning to collect data), the type of entries to use, the number of entries to return per search, the result code that should be returned for each operation, and how frequently to close and re-establish connections.

It's obviously the case that, as the primary developer for the UnboundID LDAP SDK for Java, I am more than a little biased about which SDK I prefer. However, to the best of my knowledge the way that the tool performs the tests is as fair as possible and uses the most efficient mechanism offered by each of the libraries. If anyone believes that there is a more efficient way to use any of the SDKs, then I'd be happy to hear about it and update the results accordingly.

At present, the tool provides at least some level of support for the following SDKs:

  • Apache LDAP API, version 1.0.0-M3. Although I have written code in the hope of testing this SDK, it does not appear to be fully functional at the present time. For example, when trying to perform searches with multiple threads using a separate connection per thread (which is the only way I have used it to this point), it looks like only a single thread is actually able to perform searches and all the others throw timeout exceptions. If anyone knows how to work around that, I'd be happy to hear about it. Until this problem is resolved, this tool isn't very useful for testing its performance.

  • JNDI, as is included in Java SE. For my testing, I used Java SE 1.6.0_25. JNDI is a very abstract API that has the ability to communicate using a number of protocols, and as such was not designed specifically for LDAP. Unfortunately, this means that it's not ideal for LDAP in a lot of ways. For example, it doesn't appear that JNDI provides any way to get the actual numeric LDAP result code returned by the server in response to various operations, and it also looks like it does not support bind (for the purpose of authenticating clients) as a distinct type of operation but only in the course of establishing a connection or re-authenticating before performing some other kind of operation. As such, the performance testing tool does not support bind operations, and it does not support testing with operations that return non-successful responses because the result code cannot be verified.

  • Netscape Directory SDK for Java, version 4.17 (compiled from source, as there does not appear to be a download for a pre-built version of the library). This SDK is fully supported by the performance testing tool.

  • Novell LDAP Classes for Java, also known as JLDAP, version 2009.10.07-1. This SDK is fully supported by the performance testing tool.

  • OpenDJ LDAP SDK, version 3.0.0 (snapshot build from May 28, 2011). This appears to be a fork of the OpenDS LDAP SDK that has had the package structure changed, and may have some number of additional changes as well. However, I was not able to successfully use this SDK to run any tests because the code that I used (despite identical code working for the OpenDS SDK, with the exception of changing the package names in import statements) threw an exception when trying to run, indicating that it was attempting to subclass a class that had previously been declared final. It also appeared to be missing an org.forgerock.i18n.LocalizedIllegalArgumentException class, although I worked around that problem by writing my own version of that class.

  • OpenDS LDAP SDK for Java, 0.9.0 build from May 26, 2011. This SDK is fully supported by the performance testing tool. In addition, because the API provides options for both synchronous and asynchronous connections, the "--useSynchronousMode" option is supported to request using the synchronous version of the API that does not support the use of abandon or multiple concurrent operations on the same connection, while omitting this argument will use a version that does support this capability.

  • UnboundID LDAP SDK for Java, version 2.2.0. This SDK is fully supported, including the use of the "--useSynchronousMode" option.

These tests obviously require communication with an LDAP directory server. Because the intention is not to measure the performance of the directory server but rather the SDK being used to communicate with that server, it is ideal to use something that is as fast as possible (so that the server is not a bottleneck) and that can be manipulated to give an arbitrary response for any operation. For this purpose, a custom server was created using the LDAP Listener API provided as part of the UnboundID LDAP SDK for Java. It is important to note, however, that even though this API is part of the UnboundID LDAP SDK, it can be used with any kind of client and all interaction with it was over the LDAP protocol using a socket connected over the test system's loopback interface. The UnboundID LDAP SDK did not have any advantage over any other SDK when interacting with this server.

All of the tests that I ran used Java SE 1.6.0_25 (64-bit version) on a system with a 2.67GHz 8-core Intel Core i7 CPU with 12GB of memory, running a fully-patched version of Ubuntu Linux version 11.04. A detailed description of each of the tests that I ran is provided below, along with the results that I obtained. Each test was run with the JNDI, Netscape, Novell, OpenDS, and UnboundID SDKs, using 1, 2, 4, 8, 16, 32, 64, and 128 client threads. For the OpenDS and UnboundID SDKs, tests were run using both the asynchronous and synchronous modes of operation.

Add Operation Performance

When processing add operations, performance may vary based on the size of the entry being added to the server. As such, I ran two different add performance tests: a "normal-sized" entry (an inetOrgPerson entry with 15 attributes) and a "large" entry (a groupOfUniqueNames entry with 1000 uniqueMember values).

The results I measured when running these tests were:

Add Throughput for Normal-Sized Entries

API Highest Normal-Sized Entry Add Throughput
JNDI 89,027.954 adds/sec
Netscape 98,916.582 adds/sec
Novell 86,766.964 adds/sec
OpenDS (asynchronous mode) 88,525.069 adds/sec
OpenDS (synchronous mode) 88,586.290 adds/sec
UnboundID (asynchronous mode) 142,105.659 adds/sec
UnboundID (synchronous mode) 174,665.853 adds/sec


Add Throughput for Large Entries

SDK Highest Large Entry Add Throughput
JNDI 6,472.209 adds/sec
Netscape 8,723.301 adds/sec
Novell 7,437.703 adds/sec
OpenDS (asynchronous mode) 9,454.340 adds/sec
OpenDS (synchronous mode) 9,747.643 adds/sec
UnboundID (asynchronous mode) 17,602.504 adds/sec
UnboundID (synchronous mode) 18,545.810 adds/sec


From these tests, it appears that the UnboundID LDAP SDK for Java is significantly faster than any of the other SDKs when processing add operations, and using the UnboundID LDAP SDK in synchronous mode provides a notable performance improvement over the default asynchronous mode. In contrast, the OpenDS LDAP SDK does not appear to exhibit a significant difference in add performance based on whether the asynchronous or synchronous version of the API is selected.

Search Operation Performance

As for adds, search operation performance can vary significantly based on the size of the entry being returned. As such, I ran tests search tests using the same "normal-sized" and "large" entries as for the add operation testing, and I also tested the case of returning only a single attribute in each entry. Further, because the server can return multiple entries for a single search operation, I ran tests with both operations returning a single entry and 100 identical entries. Results from those tests are provided below:

Search Throughput for 1 Tiny Entry

SDK Highest Search Throughput for 1 Tiny Entry
JNDI 54,272.423 searches/sec
Netscape 74,601.755 searches/sec
Novell 69,686.323 searches/sec
OpenDS (asynchronous mode) 73,964.204 searches/sec
OpenDS (synchronous mode) 74,572.779 searches/sec
UnboundID (asynchronous mode) 109,159.192 searches/sec
UnboundID (synchronous mode) 168,315.209 searches/sec


Search Throughput for 1 Normal-Sized Entry

SDK Highest Search Throughput for 1 Normal-Sized Entry
JNDI 46,355.770 searches/sec
Netscape 49,668.681 searches/sec
Novell 55,988.055 searches/sec
OpenDS (asynchronous mode) 55,408.763 searches/sec
OpenDS (synchronous mode) 54,923.308 searches/sec
UnboundID (asynchronous mode) 83,846.853 searches/sec
UnboundID (synchronous mode) 115,738.348 searches/sec


Search Throughput for 1 Large Entry

SDK Highest Search Throughput for 1 Large Entry
JNDI 11,045.600 searches/sec
Netscape 3,849.413 searches/sec
Novell 748.249 searches/sec
OpenDS (asynchronous mode) 10,449.903 searches/sec
OpenDS (synchronous mode) 10,374.687 searches/sec
UnboundID (asynchronous mode) 20,645.026 searches/sec
UnboundID (synchronous mode) 21,341.607 searches/sec


Search Throughput for 100 Tiny Entries

SDK Highest Search Throughput for 100 Tiny Entries
JNDI 5,749.687 searches/sec
Netscape 2,768.797 searches/sec
Novell 2,739.363 searches/sec
OpenDS (asynchronous mode) 8,295.155 searches/sec
OpenDS (synchronous mode) 8,315.379 searches/sec
UnboundID (asynchronous mode) 5,566.711 searches/sec
UnboundID (synchronous mode) 7,265.108 searches/sec


Search Throughput for 100 Normal-Sized Entries

SDK Highest Search Throughput for 100 Normal-Sized Entries
JNDI 1,983.319 searches/sec
Netscape 875.249 searches/sec
Novell 1,681.767 searches/sec
OpenDS (asynchronous mode) 1,959.581 searches/sec
OpenDS (synchronous mode) 1,917.131 searches/sec
UnboundID (asynchronous mode) 2,308.414 searches/sec
UnboundID (synchronous mode) 3,278.463 searches/sec


Search Throughput for 100 Large Entries

SDK Highest Search Throughput for 100 Large Entries
JNDI 127.716 searches/sec
Netscape 39.667 searches/sec
Novell 6.731 searches/sec
OpenDS (asynchronous mode) 117.633 searches/sec
OpenDS (synchronous mode) 117.233 searches/sec
UnboundID (asynchronous mode) 225.800 searches/sec
UnboundID (synchronous mode) 237.400 searches/sec


In this case, there is a significant variation in many of the SDKs based on the size and number of entries being returned. The UnboundID LDAP SDK is significantly faster than the other SDKs in most cases (with a notable improvement on top of that when using synchronous mode), but the OpenDS SDK is quite a bit faster than the UnboundID LDAP SDK in the case of a search returning 100 entries with only a single attribute per entry, but that is not the case for normal-sized or large entries. On the other hand, both the Netscape and Novell SDKs appear to be extremely slow when dealing with large search result entries, and the Netscape SDK is also much slower than the others for large entry sets. It is also important to note the significant drop in search performance when using JNDI for larger numbers of threads when returning a single tiny or normal-sized entry.

Modify Operation Performance

For a client SDK, modify performance has fewer variables than for either add or search operations. For a directory server, there are a number of factors, including the size of the target entry, the size of the modified attributes, and whether any of the target attributes is indexed, but none of these has an impact on the client. It is certainly the case that a modify request could update a large number of attributes and/or attribute values, but generally clients modify only one or two values at a time. As such, the only modify test run was for a modify operation replacing a single attribute value. Results for this test are:

Modify Throughput

SDK Highest Modify Throughput
JNDI 139,593.362 searches/sec
Netscape 139,362.802 searches/sec
Novell 122,199.697 searches/sec
OpenDS (asynchronous mode) 109,556.189 searches/sec
OpenDS (synchronous mode) 109,832.363 searches/sec
UnboundID (asynchronous mode) 196,411.917 searches/sec
UnboundID (synchronous mode) 242,248.797 searches/sec


Again, the UnboundID LDAP SDK is significantly faster than the other SDKs, and again, there is a significant advantage to using synchronous mode. The Novell SDK's modify performance seems to drop off significantly for higher numbers of threads.


Download a complete set of results for all tests run as an OpenDocument spreadsheet.

In most cases, the UnboundID LDAP SDK for Java is faster than all other SDKs by a wide margin, and in all cases using the UnboundID LDAP SDK for Java in synchronous mode was faster than using the SDK in the default asynchronous mode. As such, if you are using the UnboundID LDAP SDK for Java and don't need to perform asynchronous operations, then it is highly recommended that you enable synchronous mode for connections used by that application.

The only case in which the UnboundID LDAP SDK for Java was not the fastest is for a search operation in which a large number of entries were returned with only a single attribute per entry. It was the fastest for both other tests involving a large number of entries, and it was also the fastest for returning only one entry with a single attribute. I will investigate the UnboundID LDAP SDK's performance in this area to determine whether it can be improved.

The OpenDS LDAP SDK (which was started after I left OpenDS, and for which I have not participated in its development in any way) appears to be the second fastest. It was the only SDK to perform faster than the UnboundID LDAP SDK in any of the tests, and it was never the slowest of any of the tests. There does not appear to be any measurable difference in performance when using the synchronous mode versus asynchronous mode. Across all of the tests, the OpenDS LDAP SDK achieved about 56.0% of the overall performance of the UnboundID LDAP SDK in synchronous mode, and 67.8% of the performance of the UnboundID LDAP SDK in asynchronous mode.

JNDI offers middle-of-the-pack performance in most cases, but its very poor showing for searches returning a single entry with high numbers of threads may be a significant cause for concern, since this is a very common kind of operation.

The Novell SDK performance when dealing with large search result entries is very troublesome, and it is also significantly slower than all other SDKs for modify operations with a high degree of concurrency. The Netscape SDK also appears to have problems with large search result entries, and its search performance for searches returning multiple entries is a problem as well.


UnboundID LDAP SDK for Java 2.2.0

UnboundID LDAP SDK for Java 2.2.0 has just been released and is available for download from the UnboundID website or the SourceForge project page, and is also available in the Maven central repository.

The release notes provide a full overview of the changes in this release over the previous 2.1.0 version. There are several bug fixes, but some of the most notable new features include:

  • A new Minimal Edition has been introduced. The Minimal Edition is available under the same licenses as the Standard Edition and provides support for all LDAP operations, but a number of capabilities have been removed (e.g., support for SASL authentication, a number of controls and extended operations, the persistence framework, the listener framework and in-memory directory server, JNDI and Netscape SDK migration support, etc.). The primary goal of the Minimal Edition is to provide a version of the LDAP SDK with a small jar file size which is desirable for resource-constrained environments like Android applications or other embedded use. The Minimal Edition is available as a separate download, from either the UnboundID website or SourceForge project.

  • Connection pooling support has been updated to provide the ability to automatically retry operations if the first attempt fails in a way that indicates the connection may no longer be valid. In such cases, a new connection will be established (potentially to a different server, based on the ServerSet in use for the pool) and the operation will be re-attempted on that connection. This can help isolate applications from failures if one of the target directory servers is shut down, crashes, hangs, or begins behaving erratically.

  • The in-memory directory server has been updated to add support for maintaining referential integrity (e.g., so that if an entry is deleted then that user can be automatically removed from any static groups in which the user was a member), to support LDAP transactions as described in RFC 5805, and to add support for inserting an arbitrary delay before processing operations (which can be useful in simulating environments with slower response times or higher network latencies). There have also been a couple of fixes for bugs that could cause the in-memory directory server to behave incorrectly.

  • The LDAP SDK persistence framework has been updated to provide better support for searches. Previously, it was difficult to search for entries using anything but equality searches. The generate-source-from-schema tool has been updated so that it will now generate additional methods that can make it easier to perform other kinds of searches, including presence, substring (starts with, ends with, and contains), greater-or-equal, less-or-equal, and approximately-equal-to.

  • New methods have been added which make it significantly easier to interact with response controls. Each response control class now has one or more static get methods that can be used to extract and decode a response control of that type from a given response object.

  • Support for GSSAPI authentication has been significantly improved to add support for a number of new options, including the ability to indicate whether to use (or even require) a ticket cache, to specify an alternate location for the ticket cache file, and to request that the TGT be renewed. Changes have also been introduced to make it easier to access GSSAPI debugging information.

  • A new option has been added that makes it possible to automatically send an abandon request to the directory server if a client-side timeout is encountered while waiting for a response to an operation. Previously, the LDAP SDK would throw an exception but did not have any option to attempt to abandon the operation in the directory server.

  • The LDAP SDK can now use schema information (if available) in the process of normalizing and comparing DNs and RDNs. This can provide more accurate matching for DNs that use attributes in which something other than case-inexact string matching should be used.

  • The LDIF reader has been updated to provide the ability to read data from multiple files. This can be useful for cases in which the complete set of data you want to process is broken up into multiple files (e.g., representing different portions of the DIT).


UnboundID LDAP SDK for Java 2.1.0

We have just released the latest version of the UnboundID LDAP SDK for Java, version 2.1.0. You can get it from either the UnboundID website, or from the SourceForge project page.

This is a pretty significant update that has a number of changes in several areas. You can see the release notes for a fairly comprehensive overview of the changes that have been made in this version, but the major focus of the 2.1.0 release was in improving testability, and there are two new features that can significantly help with that:

  • The LDAP SDK now comes with the ability to create one or more very lightweight yet surprisingly feature-rich LDAP server instances that you can use for testing the directory-enabled applications that you are developing. Although it's not suitable for production use, it does support quite a few controls and extended operations, SASL authentication, an LDAP-accessible changelog and other features that your application might rely on. It's also got a lot of nice testing-friendly features like the ability to make assertions about the state of data in the server, the ability to create and restore point-in-time snapshots so that you can revert the server to a known state, and the ability to have multiple listeners that you can control independently for things like testing failover between servers.

  • It also provides a new LDAPTestUtils class that has lots of helpful methods for use in writing test cases for directory-enabled applications. This includes methods for generating simple entries of various types (e.g., users, groups, organizations, organizationalUnits, etc.), for verifying the contents of a directory server over LDAP, and for verifying results received from processing operations.

These new features have already made it significantly easier for my own testing in developing the LDAP SDK itself, when writing LDAP-based tools, and even in the development of the UnboundID server products like the Directory Server, Directory Proxy Server, and Synchronization Server. Hopefully, it will make it easier for you to test your own applications as well. I'll be writing additional posts in the near future that go into more detail about how you can use these new features to make it easier to test your directory-enabled applications.


UnboundID LDAP SDK for Java 2.0.1

We have just released UnboundID LDAP SDK for Java version 2.0.1. It is available for download from the UnboundID website or from the SourceForge project page, and should show up in the Maven central repository in the near future. Some of the changes in this release include:

  • The persistence framework has been updated to support object inheritance. If an object whose class is marked with the @LDAPObject annotation also has a superclass that is marked with @LDAPObject, then the annotated fields and methods of the superclass will be used in the course of converting between objects and entries.

  • The EntryValidator has been updated to fix a potential bug that could cause an exception to be thrown when processing an entry that did not include all superior classes for all of the object classes in the entry.

  • Updated the ValuePattern to provide support for back-references, which make it possible to include the same value twice in a generated string (e.g., in the pattern "uid=user.[1-100],ou=org.[ref:1],dc=example,dc=com", a random number between 1 and 100 will be chosen for the user number, and then that same value will also be used for the organization number). In addition, a bug has been fixed that could interfere with the ability to use the ValuePattern to obtain content from files on Windows systems when the provided path included a drive letter.

  • The argument parsing framework has been updated to include a new ArgumentListArgument, in which the values of the argument list argument are themselves argument strings (e.g., "--arg1Name arg1value --arg2Name --arg3Name arg3value"), and the argument will have its own parser to perform the validation. Also, the StringArgument has been updated to make it possible to specify a regular expression that can be used to constrain the set of allowed values.

  • When creating an LDAPConnectionPool using a ServerSet, it is now possible to request that zero connections be initially created. If this is done, then the pool will not have any connections established until the first time an attempt is made to use it.

  • The Filter class has been updated to expose static encodeValue methods, which may be used to properly escape a value for use in the string representation of a filter. This should only be used if you are manually creating a filter string representation, and is not necessary if you are creating a filter by providing attribute type and value objects.

  • Debug messages generated by the LDAP SDK have been improved so that they can be more useful, especially for those messages related to LDAP communication in an environment in which many connections may be in use at the same time. Each debug message about LDAP communication will include information about the connection used for that communication and the LDAP message ID for the request or response. Further, result messages will now include the protocol op type.


The Movies of 2010

I've mentioned in the past that I like to watch movies. I watch a lot of movies at home (on DVD/Blu-Ray, TV, Netflix streaming, Amazon Video on Demand, Hulu, etc.), but the home experience just can't match seeing a movie on the big screen. Fortunately, I live in Austin, TX, which is the home of the Alamo Drafthouse cinemas, some significant film festivals (including SxSW, Fantastic Fest, and Austin Film Festival), and a decent arthouse theater. This means that I have plenty of opportunities to see a wide variety of movies, and I take advantage of them. Last year, I saw a little over 100 movies in theaters. Back in 2007, I saw nearly 130. But this year, a combination of factors made the number a bit higher. I expanded the range of theaters I attended on a regular basis, I was a badge holder at Fantastic Fest for the first time (and took full advantage of that), and I started watching less TV. But all that together still doesn't quite account for my final in-theater movie count for the year. I really don't know how, and I'm kind of embarrassed to admit it, but I attended 512 theater showings in 2010.

Some basic statistics about the movies that I saw this year:

  • I saw 254 different movies during or before what I consider to be their initial theatrical run. I wrote up my thoughts on nearly all of them at http://www.witamis.com/.
  • An additional 43 showings were repeats of those first-run movies.
  • An additional 3 showings were repeats of first-run movies I first saw in 2009.
  • I saw 97 additional movies that were not in their first theatrical run but that I had never seen before.
  • I saw 39 different movies at Fantastic Fest, which is the most a non-press attendee was able to attend.
  • I saw 26 different movies at Austin Film Festival, which I believe to be the most an attendee was able to attend.
  • 98 of the showings (nearly 1 in 5) included someone directly involved with that movie (e.g., an actor, writer, director, producer, or other crew member) in attendance.
  • I attended 20 other showings with some other notable guest who wasn't directly involved in making the movie but added significantly to the experience.
  • I attended 20 Master Pancake presentations (a showing in which the movie was mocked by comedians in the style of Mystery Science Theater 3000).
  • I attended 22 quote-along presentations, in which the audience is encouraged to recite key lines from the movie at the same time it's said on screen.
  • I attended 39 Terror Tuesday showings and 20 Weird Wednesday showings.
  • I attended 2 Alamo Drafthouse Rolling Roadshow presentations, in which a classic movie was shown in a symbolic location.
  • 61 of the movies I saw were primarily in a language other than English.

My Favorite Movie of the Year

This one is easy: Scott Pilgrim vs. the World. This is the only movie that I've ever seen twice in the theater on the same day, and it's the only movie I've seen in the theater seven times during its initial run. The first time I saw the movie was a special presentation with writer/director Edgar Wright, co-writer Michael Bacall, and cast members Michael Cera, Mary Elizabeth Winstead, Anna Kendrick, Brandon Routh, and Jason Schwartzman, and that was my favorite single showing and favorite Q&A of the year. I got the Blu-Ray/DVD set on the day it was released and have seen it many times, including all four commentaries, and most of the other extra features. I bought the soundtrack and have listened to it many times. I bought and read all six graphic novels on which it was based, and it may be somewhat sacrilegious to say but I actually think that the movie is better than the comic (despite a line in the movie which states otherwise).

I was extremely excited about this movie before it came out, and my expectations were greatly exceeded when I saw it. The more times I see it, the more I'm impressed by the tiny details that the vast majority of people will miss. It has a truly unique visual style that's impossible to adequately describe without actually seeing it. It's an absolute travesty that it didn't even make back its budget, but for some reason people just didn't go to see it. Hopefully it will make a ton of money from people buying it on DVD and/or Blu-Ray.

The Overall Best Movie of the Year

This one is also easy: Bedevilled. I saw it at Fantastic Fest and was blown away. I love Korean vengeance movies, and this is one of the best. It's the first effort from director Jang Cheol-so (who was in attendance to introduce the movie and provide a Q&A afterward), and I can't wait to see what else he has in store in the future.

Although neither The Man From Nowhere nor Kidnapped were quite as good as Bedevilled, they were both better than any other new movie I saw in 2010. Incidentally, I also saw both of them at Fantastic Fest.

The Best Mainstream Movies of the Year

Unfortunately, many of the movies that I consider among the best of the year were ones that only played at arthouse cinemas (and were therefore available only to people fortunate enough to live in a city with such a theater) or film festivals (and were therefore only available to an even smaller population). However, there were still a number of very good movies that I consider to be more "mainstream" and widely available. In my opinion, the following are the ten best movies that grossed at least 10 million dollars at the US box office and that I first had the opportunity to see this year:

There are a couple of honorable mentions that would have been on this list if they hadn't fallen just short of the $10M cutoff. They include 127 Hours (which made $9.86M) and Get Low (which made $9.11M). And if The King's Speech had been released a little earlier it might have also crossed the threshold before the end of the year. But I am both shocked and happy that The Girl with the Dragon Tattoo did make the cut with just over $10M in the US (and another $100M worldwide) despite a runtime over 2 and a half hours and being entirely in Swedish with English subtitles. Incidentally, it is available for streaming on Netflix, along with its sequel The Girl Who Played with Fire, and the third movie of the triology The Girl Who Kicked the Hornet's Nest should be available for streaming by the end of January 2011.

Other Movies Worth Mentioning

All of the following movies made far less than they should have and/or weren't as widely available as they deserved to be, but are still worth checking out if you get the chance. Many of them are already available on DVD and/or Blu-Ray, and a couple of them (Mother and The Good, the Bad, the Weird) are available for streaming on Netflix.


UnboundID LDAP SDK for Java 2.0.0

We have just released the 2.0.0 version of the UnboundID LDAP SDK for Java. It is available for download on the UnboundID LDAP SDK product page or on the SourceForge project page.

This is a major new release that includes significant new functionality, including:

  • A persistence API that makes it easy to interact with entries in an LDAP directory as Java objects. Annotations are used to establish mappings between LDAP attributes and content in Java objects, and the data is stored in a format that makes it easily accessible by other APIs as well. Tools are provided to help you generate source code from existing schema, or generate LDAP schema from annotated source.

  • A listener API that makes it easy to develop applications that can listen for communication from LDAP clients (i.e., applications which act as an LDAP server). A new tool is provided which makes use of this API to act as a simple LDAP proxy that can intercept and decode LDAP communication as it passes between an LDAP client and server.

  • New types of command-line arguments, including Boolean value arguments, duration arguments, and search scope arguments. The file argument type has also been updated to allow you to specify a base directory for relative paths so that non-absolute paths will be relative to a specified location.

There are also a few other minor enhancements and bug fixes.


The Movies of Fantastic Fest 2010

Aside from writing code, one of my favorite pastimes is watching movies, and that's something that I do quite a lot. So far this year, I've seen just under 350 movies in a theater, and I have no idea how many I've seen on TV, DVD/Blu-Ray, Netflix, Amazon, Hulu, or some other format.

From September 23-30, I attended Fantastic Fest, a film festival sponsored by the Alamo Drafthouse and Ain't It Cool News that focuses primarily on horror, thriller, and sci-fi movies. During this eight-day period, I saw a total of 39 different movies (five movies per day for the first week, and then four on the last day). They were all movies I had never seen before, and with the exception of the 1960 version of The Housemaid (which I wanted to see before watching the 2010 remake) they were all new movies that haven't had any kind of wide release (and in many cases, they won't ever get one).

Below, you will find a list of all the new movies that I saw during this festival, in order of what I consider to be most enjoyable to least enjoyable. Each is linked to a very short write-up in which I provide a description about the movie and what I thought about it.

  1. Bedevilled (10/10)
  2. The Man From Nowhere (9/10)
  3. Kidnapped (9/10)
  4. A Somewhat Gentle Man (9/10)
  5. Undocumented (9/10)
  6. Drones (9/10)
  7. Golden Slumber (8/10)
  8. Bunraku (8/10)
  9. Richard Garriott: Man On a Mission (8/10)
  10. Nevermore (8/10)
  11. Julia's Eyes (8/10)
  12. The Housemaid (8/10)
  13. Mother's Day (8/10)
  14. A Horrible Way to Die (7/10)
  15. Sound of Noise (7/10)
  16. Carancho (7/10)
  17. Red Hill (7/10)
  18. I Spit On Your Grave (7/10)
  19. Stake Land (7/10)
  20. Fatso (7/10)
  21. Hatchet 2 (7/10)
  22. In The Attic (7/10)
  23. Rammbock (7/10)
  24. Cold Fish (6/10)
  25. Agnosia (6/10)
  26. Outrage (6/10)
  27. Corridor (6/10)
  28. Rubber (6/10)
  29. Primal (6/10)
  30. Fire of Conscience (6/10)
  31. Transfer (6/10)
  32. Norwegian Ninja (6/10)
  33. The Dead (6/10)
  34. Gallants (5/10)
  35. 14 Blades (5/10)
  36. We Are What We Are (5/10)
  37. The Violent Kind (4/10)
  38. Tetsuo: The Bullet Man (1/10)

UnboundID LDAP SDK for Java 1.1.6

I have just made a new 1.1.6 release of the UnboundID LDAP SDK for Java available for download. You can get it on either the UnboundID LDAP SDK product page or on the SourceForge project page. It should also be available in the Maven Central Repository in the near future (probably within the next day or two).

There are only a couple of minor changes in this release over the previous 1.1.5 version, including:

  • I corrected an error in the documentation around the use of the content synchronization request control that incorrectly recommended using a response timeout of -1 rather than 0. A value of 0 indicates that no timeout should be used, while a value of -1 means that the connection-level timeout (which is 5 minutes by default) should be used.

  • I fixed a problem in the way that the string representation of RDNs was generated when a DN was parsed from a string. Previously, some extra spaces could have been removed. The resulting string representation was technically equivalent to the version that was originally provided, but the original formatting should have been preserved.

  • I updated the source for the Android LDAP client to match what was published as version 1.1 in the Android market.


Minor Updates to the Android LDAP Client

I have just uploaded a new version of the UnboundID LDAP client to the Android market. It's kind of embarrassing that it took this long to put out such a minor update, but I had visions of a much more significant update that I never got around to coding and I kept putting off some simple stuff. Hopefully the minor improvements are worth the update, and maybe I'll get around to the bigger version at some point in the future.

The changes included in this update are:

  • I've updated the application to use the latest version of the LDAP SDK. This includes several bug fixes over the version used in the previous release of the LDAP client, including some fixes around SSL and StartTLS.

  • It is now more liberal with what it considers a user entry (and will therefore allow special treatment for things like telephone numbers, e-mail addresses, postal addresses, etc.). Previously, it would only work for entries with an object class of "person". It will now work for any entry that has a "cn" attribute and at least one of the following additional attributes: mail, mailAlternateAddress, telephoneNumber, homePhone, mobile, pager, and facsimileTelephoneNumber.

  • It should now be usable on devices with any screen size. Previously, it was only available for devices with what was considered a "normal" screen size, and it might not have been available on some devices with a very low resolution, or on devices with much higher resolutions. I don't know how it looks on any of those other devices, but hopefully it's at least usable.

  • It should now be possible to move the application to the SD card on devices running Android 2.2.

I've tested the application in the emulator for Android versions 1.5 and 2.2, and on actual phones running Android 1.6, 2.1, and 2.2, and I haven't encountered any problems on any of them. If you run into a problem, please let me know.