Yesterday Tim Anderson asked a question: ”Why doesn’t Adobe’s AIR dev guide mention SQLite?” As the author of the ”Working with local SQL databases” chapter in the AIR Developers Guide that Tim refers to, as well as the related sections of the AIR language reference, I guess I know better than most people the answer to that question. Unfortunately, although it was a consciously considered decision that underwent a reasonable amount of discussion, I can’t promise that the reasoning is absolutely irrefutable logic that will stand the test of time. But then, if it was, Tim probably wouldn’t have asked the question =)
Note: This post won’t make as much sense unless you read Tim’s post first. So I’ll pause for a moment while you go read it – don’t worry, it’s not nearly as long as this one =)
As suggested by John Dowdell in the comments on Tim’s post, the biggest reason why SQLite isn’t mentioned by name in the documentation is a somewhat complex matter involving the timing of when the feature was developed, deadlines for when the documentation had to be done in order to include it in the beta release, and the timing of when Google Gears was announced.
Background: Development schedules, marketing announcements, and the realities of human limits
If you don’t mind, I’ll start off with a bit of background/context on the timing of the local SQL database feature being developed and the agreement with Google.
Note: Admittedly I’m making educated guesses on the timing described here, based on things I observed but without actually being involved in the conversations between Adobe and Google.
Before any sort of agreement about synchronized APIs and common code had been reached, perhaps before it had even started, Adobe, and presumably Google, had both separately developed most of their respective implementations of database functionality based on SQLite. Naturally, at the point that they agreed to make their APIs agree with each other, both companies could have chosen to postpone releasing their implementations until the final, synchronized API was finished. Not surprisingly, both companies felt it was more important to get something out to developers sooner (even if it wasn’t the “synchronized API”) rather than later when things were final. From a public visibility perspective, Google made their announcement first, so they have the benefit of being perceived as the “original” and I expect many developers expected the AIR API to be identical when it was released (about a week and a half later, it’s important to note).
I wasn’t involved in any of the planning or discussion surrounding the AIR-Google Gears “synchronized API” decision, and in fact I didn’t know anything about it until I read the public announcements. (If you can, imagine my reaction when I read that the APIs were going to be aligned – since I had just finished the API reference documentation the previous week! =) The fact that I wasn’t involved in the conversations doesn’t bother me and makes sense to me – there wasn’t really any compelling reason why I should have been involved in those conversations – but obviously that had a direct impact on my ability to anticipate the consequences of that announcement while I was writing the documentation. By the time the announcement was made, it was too late to go back and change the documentation, and definitely too late for us to actually change the API.
With that background, let me address some of the specific points that Tim raises. He asks the question, “why doesn’t Adobe’s AIR dev guide mention SQLite?”, and gives three points about why he isn’t pleased that SQLite isn’t mentioned:
1. “It stinks”
Of course, what I’ve said up to this point suggests that, were it not for the Google Gears announcements, Adobe might not have ever announced that we are using SQLite. I can’t speak for alternative futures, of course, but I think it’s possible that Adobe would never have mentioned SQLite in the documentation. Does that “stink” as Tim suggests? Is it discourteous? I can definitely understand the sentiment behind those expressions. I’m a firm believer in “credit where it’s due.” When I wrote the documentation, I certainly didn’t mean it as a discourtesy or to minimize my opinion of the value of SQLite that SQLite was not mentioned. I think SQLite is a great tool, and I’m very glad it exists, and that the authors have been so generous with the license.
On the other had, as Tim notes, Adobe has no legal obligation to acknowledge that they included source code from SQLite. Nor does Adobe have any obligation to acknowledge that a significant part of the documentation on the SQL dialect was adapted from the SQLite documentation, because the authors were even kind enough to secure public domain releases for the documentation – although I will readily admit that I did in fact adapt portions of the SQLite documentation – which was very helpful because it saved me a lot of duplicate effort!
So, were we intentionally trying to hide the fact that it was SQLite? No, although we chose not to advertise the fact either, for a few reasons:
- Our intention was to provide sufficient documentation so that a developer wouldn’t have any need to look at the SQLite documentation. (In fact, due to the short time frame for getting things done, one idea that was considered and rejected was just to point developers to the SQLite documentation for the first beta release.) With that decision made, we realized that there wasn’t any absolute need to make reference to SQLite.
- In many respects, the choice of SQLite was originally considered to be an “implementation detail” – something that developers didn’t really need to know or care about. Obviously in the wake of the Google Gears announcements, there is now more reason why it does matter (to some developers) that Adobe chose to use SQLite.
In fact, when you get down to it, the choice of SQLite could still be considered an “implementation detail.” After all, consider what happens in the future when AIR version 5 or 10 is released. What if SQLite is no longer the best choice for an embedded database? Should AIR developers be limited by the fact that AIR is absolutely tied to SQLite? As long as the API continues to work, and the older file-format databases are supported, and older applications continue to run, how much does it matter whether the embedded database is SQLite or something else?
Obviously that’s not a highly likely scenario. From a practical standpoint, even if another database engine were used, it would be pretty involved to ensure backwards compatibility, both with the API and especially with the supported SQL dialect.
In summary, when the time came to decide whether to explicitly mention SQLite in the documentation, it came down to this. As we all know, it’s common practice in the software industry, including at Adobe, that when a certain aspect of a feature is considered an “implementation detail” and “not important to developers,” that aspect of the feature isn’t described in the documentation. Sometimes developers find out anyway, either through their own experimentation, or through experience, or whatever. Even if Adobe didn’t announce that the AIR local database engine was based on SQLite, it’s highly likely that it would have come out sooner rather than later. That’s fine – developers can choose to use the knowledge they acquire, and Adobe can choose to change things that aren’t documented.
As I’ve said, at the time the documentation was being written and finalized, the word I was going on was that the fact that the engine uses SQLite is an implementation detail. I gave some thought to whether we should explicitly mention that the database uses SQLite – certainly there were advantages and disadvantages either way. The issue was discussed by me, members of management, and the lead engineer for the feature. In the end, however, for the reasons listed above, and without any knowledge (at least on my part) of the very-soon-to-be-forthcoming Google Gears alignment announcement, we decided that the priorities fell on the side of not stating that the database uses SQLite.
Having said all that, as a result of the announcements relating to Adobe AIR and Google Gears, it is public knowledge that Adobe is using SQLite. Although it may not make sense to anyone but me, that announcement “changes the balance” of priorities in my mind, so in fact at this point I agree with Tim that the benefits of mentioning in the documentation that the database uses SQLite (giving credit where it’s due, and guiding developers to other SQLite tools that might help them) outweigh the potential downsides of stating it in the documentation (the unlikely but can’t-rule-it-out-completely possible future case in which we no longer use SQLite for the underlying database engine, and the possibility of distracting/confusing developers who read the SQLite documentation and find something that’s not supported in AIR and wonder why).
Even as I write this, I acknowledge that my stated set of reasons, and this description of trying to find the best balance between the tradeoffs of “officially” disclosing information versus “officially” keeping information “undocumented” sounds very fuzzy. And I agree that the value of giving credit to the authors of SQLite is a compelling reason for stating in the documentation that AIR uses SQLite. I’m afraid I can’t completely communicate how fast and furiously we were working to get this feature (and the associated documentation) done for the beta, and consequently how we tried to make the best choices we could in a limited time frame. And again, without the smallest clue of the then-forthcoming Google Gears announcement.
Fortunately, as Tim points out, “this is all beta, of course, so it can change.” So, Tim, allow me to tell you that your feedback has helped us to realize that the value of giving credit to the authors of SQLite outweighs any potential drawbacks, and I will shortly be adding explicit mention of the fact that the runtime uses SQLite for its local SQL database functionality. A week ago I asked for feedback on the SQL documentation, and while this wasn’t exactly what I had in mind, it is nonetheless appreciated and hopefully it’s clear that it has been taken into consideration.
2. “It’s unhelpful”
Tim makes the suggestion that it would be helpful to “set out exactly how AIR’s SQLite differs from the standard build.” I think that sounds like a useful suggestion; I’ve made a note to research the topic and see how difficult it would be to compile that information, and how we could usefully incorporate it into the documentation. I admit that the thought didn’t even occur to me (which doesn’t surprise me – I have some good ideas sometimes, but I certainly don’t ever have all the good ideas =).
Although I acknowledge that it sounds like an idea with merit, the devil’s advocate in me impels me to admit that specifying how AIR’s SQLite implementation is different from other SQLite implementations isn’t as much of a priority as some of the other tasks that I’ve got in the lineup. It turns out that there were several things that I wanted to write, and we just didn’t have time to get them all in for the beta release. Since the SQLite library is designed for use as an embedded database, and the target audience (developer-wise) for AIR is developers with web development experience, chances are the majority of the target audience for AIR hasn’t ever programmed for SQLite. That means that documentation comparing AIR’s implementation to other implementations will probably only be useful for a minority of AIR developers – which unfortunately is something that we have to take into account when we’re scheduling the “features” that are chosen to be included in the documentation.
Nevertheless, I do think that it’s a good suggestion, and in some ways I think that it will be useful to more advanced developers whether they’ve used SQLite or not, so I’ll see what I can do.
3. Skepticism that the AIR and Google Gears APIs will align
Hopefully my earlier background information has made it clear that the reason the APIs don’t align right now is really a matter of scheduling, engineering effort, design effort, and a desire on the part of both companies to get their technologies into the hands of developers, and not any sort of conspiracy. I will say that I know for certain that the effort to align the APIs is still going on.
“A big hole in the AIR sandbox”
Tim makes a side note at this point, that the difference between the AIR approach to SQLite and the Google Gears approach has some security implications. This is an important point, so I’ve written a separate post in which I’ve discussed why the two systems use different models, and how to address some of the potential security issues.
Summary: Tim’s wish list
Going back to Tim, he concludes with a “wish list” of three things he’d like to see happen in the AIR documentation and implementation:
- Proper credit for SQLite in the docs.
- Use the Gears code - full text search could be very useful - and deliver on the promise of aligning the API.
- Failing that, set out exactly how AIR’s SQLite differs from the standard build.
I’ll consider these one at a time:
Proper credit for SQLite in the docs
As Tim and I have both mentioned, SQLite is in the public domain, so in fact from a legal definition Adobe is already giving “proper credit” (that is, no credit required) for SQLite. I’m certain that’s not what Tim had in mind, and I’ve also mentioned that I believe it’s worthwhile to give credit to the authors of SQLite. As I’ve also said, I’m planning to add in at least a couple of prominent places a statement that AIR uses SQLite for its database functionality. If that’s not what you had in mind, Tim, please let me know.
Use the Gears code - full text search could be very useful - and deliver on the promise of aligning the API.
I’m 100% certain Adobe is not going to “use the [Google] Gears code,” at least not wholesale – the two products have already been developed in isolation much too much for that to work (in my opinion – but of course I’m not the engineer). Nevertheless, I agree that full text search could be very useful – I would love to see it make it in to AIR as well – and as I’ve said here and also last week, Adobe and Google are continuing to work on aligning their database APIs.
Failing that, set out exactly how AIR’s SQLite differs from the standard build.
This statement is actually a bit misleading, I think, since I’m certain Google Gears doesn’t use a 100% “standard”, out of the box SQLite build. (For that matter, I’m not sure what a “standard” SQLite build would be, since the whole point of SQLite is to be an embedded database, built into another product, and not to stand on its own.)
All that notwithstanding, as I’ve mentioned I have now assigned myself the task of researching the changes that we’ve made from what someone who’s used SQLite in another product might expect, and (time permitting) to add a section to the documentation describing those differences.
Conclusion
I realize this whole post is long, and meanders quite a bit, and probably doesn’t show the degree of concrete explanation (or completely serious consideration) that might be desirable. I believe that we made the decisions we made for good reasons, using the information we had available at the time. I’m also glad that we’ve only released a beta and not the final version yet, and I’m glad that Tim’s post caused me and others in Adobe to sit up and take notice, and reconsider some of our decisions in light of the new information that’s become available since the time when we previously made them.
And once again, if anyone has thoughts or suggestions about the AIR local SQL database documentation, please don’t hesitate to share them. I’ll try not to post an essay in response to each one =)