Why doesn’t Adobe’s AIR dev guide mention SQLite? A response to Tim Anderson

Posted June 19, 2007 11:11 pm
Filed under: AIR, Articles by Paul, Opinions, Writing, local SQL database

Yesterday Tim Anderson asked a question: “Why doesn’t Adobe’s AIR dev guide mention SQLite?” As the author of the “Working with local SQL databases” chapter in the AIR Developers Guide that Tim refers to, as well as the related sections of the AIR language reference, I guess I know better than most people the answer to that question. Unfortunately, although it was a consciously considered decision that underwent a reasonable amount of discussion, I can’t promise that the reasoning is absolutely irrefutable logic that will stand the test of time. But then, if it was, Tim probably wouldn’t have asked the question =)

Note: This post won’t make as much sense unless you read Tim’s post first. So I’ll pause for a moment while you go read it — don’t worry, it’s not nearly as long as this one =)

As suggested by John Dowdell in the comments on Tim’s post, the biggest reason why SQLite isn’t mentioned by name in the documentation is a somewhat complex matter involving the timing of when the feature was developed, deadlines for when the documentation had to be done in order to include it in the beta release, and the timing of when Google Gears was announced.

Background: Development schedules, marketing announcements, and the realities of human limits

If you don’t mind, I’ll start off with a bit of background/context on the timing of the local SQL database feature being developed and the agreement with Google.

Note: Admittedly I’m making educated guesses on the timing described here, based on things I observed but without actually being involved in the conversations between Adobe and Google.

Before any sort of agreement about synchronized APIs and common code had been reached, perhaps before it had even started, Adobe, and presumably Google, had both separately developed most of their respective implementations of database functionality based on SQLite. Naturally, at the point that they agreed to make their APIs agree with each other, both companies could have chosen to postpone releasing their implementations until the final, synchronized API was finished. Not surprisingly, both companies felt it was more important to get something out to developers sooner (even if it wasn’t the “synchronized API”) rather than later when things were final. From a public visibility perspective, Google made their announcement first, so they have the benefit of being perceived as the “original” and I expect many developers expected the AIR API to be identical when it was released (about a week and a half later, it’s important to note).

I wasn’t involved in any of the planning or discussion surrounding the AIR-Google Gears “synchronized API” decision, and in fact I didn’t know anything about it until I read the public announcements. (If you can, imagine my reaction when I read that the APIs were going to be aligned — since I had just finished the API reference documentation the previous week! =) The fact that I wasn’t involved in the conversations doesn’t bother me and makes sense to me — there wasn’t really any compelling reason why I should have been involved in those conversations — but obviously that had a direct impact on my ability to anticipate the consequences of that announcement while I was writing the documentation. By the time the announcement was made, it was too late to go back and change the documentation, and definitely too late for us to actually change the API.

With that background, let me address some of the specific points that Tim raises. He asks the question, “why doesn’t Adobe’s AIR dev guide mention SQLite?”, and gives three points about why he isn’t pleased that SQLite isn’t mentioned:

1. “It stinks”

Of course, what I’ve said up to this point suggests that, were it not for the Google Gears announcements, Adobe might not have ever announced that we are using SQLite. I can’t speak for alternative futures, of course, but I think it’s possible that Adobe would never have mentioned SQLite in the documentation. Does that “stink” as Tim suggests? Is it discourteous? I can definitely understand the sentiment behind those expressions. I’m a firm believer in “credit where it’s due.” When I wrote the documentation, I certainly didn’t mean it as a discourtesy or to minimize my opinion of the value of SQLite that SQLite was not mentioned. I think SQLite is a great tool, and I’m very glad it exists, and that the authors have been so generous with the license.

On the other had, as Tim notes, Adobe has no legal obligation to acknowledge that they included source code from SQLite. Nor does Adobe have any obligation to acknowledge that a significant part of the documentation on the SQL dialect was adapted from the SQLite documentation, because the authors were even kind enough to secure public domain releases for the documentation — although I will readily admit that I did in fact adapt portions of the SQLite documentation — which was very helpful because it saved me a lot of duplicate effort!

So, were we intentionally trying to hide the fact that it was SQLite? No, although we chose not to advertise the fact either, for a few reasons:

In summary, when the time came to decide whether to explicitly mention SQLite in the documentation, it came down to this. As we all know, it’s common practice in the software industry, including at Adobe, that when a certain aspect of a feature is considered an “implementation detail” and “not important to developers,” that aspect of the feature isn’t described in the documentation. Sometimes developers find out anyway, either through their own experimentation, or through experience, or whatever. Even if Adobe didn’t announce that the AIR local database engine was based on SQLite, it’s highly likely that it would have come out sooner rather than later. That’s fine — developers can choose to use the knowledge they acquire, and Adobe can choose to change things that aren’t documented.

As I’ve said, at the time the documentation was being written and finalized, the word I was going on was that the fact that the engine uses SQLite is an implementation detail. I gave some thought to whether we should explicitly mention that the database uses SQLite — certainly there were advantages and disadvantages either way. The issue was discussed by me, members of management, and the lead engineer for the feature. In the end, however, for the reasons listed above, and without any knowledge (at least on my part) of the very-soon-to-be-forthcoming Google Gears alignment announcement, we decided that the priorities fell on the side of not stating that the database uses SQLite.

Having said all that, as a result of the announcements relating to Adobe AIR and Google Gears, it is public knowledge that Adobe is using SQLite. Although it may not make sense to anyone but me, that announcement “changes the balance” of priorities in my mind, so in fact at this point I agree with Tim that the benefits of mentioning in the documentation that the database uses SQLite (giving credit where it’s due, and guiding developers to other SQLite tools that might help them) outweigh the potential downsides of stating it in the documentation (the unlikely but can’t-rule-it-out-completely possible future case in which we no longer use SQLite for the underlying database engine, and the possibility of distracting/confusing developers who read the SQLite documentation and find something that’s not supported in AIR and wonder why).

Even as I write this, I acknowledge that my stated set of reasons, and this description of trying to find the best balance between the tradeoffs of “officially” disclosing information versus “officially” keeping information “undocumented” sounds very fuzzy. And I agree that the value of giving credit to the authors of SQLite is a compelling reason for stating in the documentation that AIR uses SQLite. I’m afraid I can’t completely communicate how fast and furiously we were working to get this feature (and the associated documentation) done for the beta, and consequently how we tried to make the best choices we could in a limited time frame. And again, without the smallest clue of the then-forthcoming Google Gears announcement.

Fortunately, as Tim points out, “this is all beta, of course, so it can change.” So, Tim, allow me to tell you that your feedback has helped us to realize that the value of giving credit to the authors of SQLite outweighs any potential drawbacks, and I will shortly be adding explicit mention of the fact that the runtime uses SQLite for its local SQL database functionality. A week ago I asked for feedback on the SQL documentation, and while this wasn’t exactly what I had in mind, it is nonetheless appreciated and hopefully it’s clear that it has been taken into consideration.

2. “It’s unhelpful”

Tim makes the suggestion that it would be helpful to “set out exactly how AIR’s SQLite differs from the standard build.” I think that sounds like a useful suggestion; I’ve made a note to research the topic and see how difficult it would be to compile that information, and how we could usefully incorporate it into the documentation. I admit that the thought didn’t even occur to me (which doesn’t surprise me — I have some good ideas sometimes, but I certainly don’t ever have all the good ideas =).

Although I acknowledge that it sounds like an idea with merit, the devil’s advocate in me impels me to admit that specifying how AIR’s SQLite implementation is different from other SQLite implementations isn’t as much of a priority as some of the other tasks that I’ve got in the lineup. It turns out that there were several things that I wanted to write, and we just didn’t have time to get them all in for the beta release. Since the SQLite library is designed for use as an embedded database, and the target audience (developer-wise) for AIR is developers with web development experience, chances are the majority of the target audience for AIR hasn’t ever programmed for SQLite. That means that documentation comparing AIR’s implementation to other implementations will probably only be useful for a minority of AIR developers — which unfortunately is something that we have to take into account when we’re scheduling the “features” that are chosen to be included in the documentation.

Nevertheless, I do think that it’s a good suggestion, and in some ways I think that it will be useful to more advanced developers whether they’ve used SQLite or not, so I’ll see what I can do.

3. Skepticism that the AIR and Google Gears APIs will align

Hopefully my earlier background information has made it clear that the reason the APIs don’t align right now is really a matter of scheduling, engineering effort, design effort, and a desire on the part of both companies to get their technologies into the hands of developers, and not any sort of conspiracy. I will say that I know for certain that the effort to align the APIs is still going on.

“A big hole in the AIR sandbox”

Tim makes a side note at this point, that the difference between the AIR approach to SQLite and the Google Gears approach has some security implications. This is an important point, so I’ve written a separate post in which I’ve discussed why the two systems use different models, and how to address some of the potential security issues.

Summary: Tim’s wish list

Going back to Tim, he concludes with a “wish list” of three things he’d like to see happen in the AIR documentation and implementation:

  • Proper credit for SQLite in the docs.
  • Use the Gears code - full text search could be very useful - and deliver on the promise of aligning the API.
  • Failing that, set out exactly how AIR’s SQLite differs from the standard build.

I’ll consider these one at a time:

Proper credit for SQLite in the docs

As Tim and I have both mentioned, SQLite is in the public domain, so in fact from a legal definition Adobe is already giving “proper credit” (that is, no credit required) for SQLite. I’m certain that’s not what Tim had in mind, and I’ve also mentioned that I believe it’s worthwhile to give credit to the authors of SQLite. As I’ve also said, I’m planning to add in at least a couple of prominent places a statement that AIR uses SQLite for its database functionality. If that’s not what you had in mind, Tim, please let me know.

Use the Gears code - full text search could be very useful - and deliver on the promise of aligning the API.

I’m 100% certain Adobe is not going to “use the [Google] Gears code,” at least not wholesale — the two products have already been developed in isolation much too much for that to work (in my opinion — but of course I’m not the engineer). Nevertheless, I agree that full text search could be very useful — I would love to see it make it in to AIR as well — and as I’ve said here and also last week, Adobe and Google are continuing to work on aligning their database APIs.

Failing that, set out exactly how AIR’s SQLite differs from the standard build.

This statement is actually a bit misleading, I think, since I’m certain Google Gears doesn’t use a 100% “standard”, out of the box SQLite build. (For that matter, I’m not sure what a “standard” SQLite build would be, since the whole point of SQLite is to be an embedded database, built into another product, and not to stand on its own.)

All that notwithstanding, as I’ve mentioned I have now assigned myself the task of researching the changes that we’ve made from what someone who’s used SQLite in another product might expect, and (time permitting) to add a section to the documentation describing those differences.

Conclusion

I realize this whole post is long, and meanders quite a bit, and probably doesn’t show the degree of concrete explanation (or completely serious consideration) that might be desirable. I believe that we made the decisions we made for good reasons, using the information we had available at the time. I’m also glad that we’ve only released a beta and not the final version yet, and I’m glad that Tim’s post caused me and others in Adobe to sit up and take notice, and reconsider some of our decisions in light of the new information that’s become available since the time when we previously made them.

And once again, if anyone has thoughts or suggestions about the AIR local SQL database documentation, please don’t hesitate to share them. I’ll try not to post an essay in response to each one =)

You can leave a comment, or trackback from your own site.

17 Comments so far

  1. June 20th, 2007 at 6:42 am
    andrea is reported to have said:

    We all know Apollo/Air is a work in progress and developing platform, the documentation I studied so far is way better than I expected. I was shocked when I learned Adobe chose to include SQLite in Air, this is what completely turned my interest from pure Flex2/AS3 to Air as a complete platform.

    This just to say you’re doing a great work at Adobe Labs, thank you.
    andrea

  2. June 20th, 2007 at 11:38 am
    Chris Seahorn is reported to have said:

    Great article!

    Reading Tim’s article his first paragraph would tend to make me believe he has an axe to grind and immediately gives the impression that Adobe AIR is a confusing mess at best. Following that it’s hard to take his offense at Adobe’s handling of the SQLite additions seriously because after that opening I can only assume he’s

    1. Easily confused
    2. Easily offended

    I don’t think anyone familiar with AIR or following the progression of the platform is not aware of the fact that SQLite is the database of choice (right now). A simple Google search will attest to that so how (in beta) Tim feels SQLite’s honor was so besmirched by Adobe is beyond me.

    I think your responses to his wish list items are :

    1. Honest
    2. Practical
    3. Make perfect sense
    4. Truthful
    5. Will be accomplished

    So when they are, I almost can’t wait to see what other things become fodder for cheap shots by people with hidden agendas.

    Love the blog BTW :)

    Chris

  3. June 21st, 2007 at 3:20 am
    Tim Andeson is reported to have said:

    Chris,

    Well, perhaps I am easily confused. I have no axe to grind though.

    AIR has two personalities. One is for browser apps, the other is for Flash apps (coded with Flex), both running on the desktop. Hence two separate SDKs. That’s the fact I wanted to reflect in my opening paragraph.

    I’d be grateful if you could clarify why this or any other part of my post is a “cheap shot”, also please explain the hidden agenda you allege?

    I agree though that Paul’s response is really helpful and encouraging.

    Tim

  4. June 21st, 2007 at 4:36 am
    Eduardo is reported to have said:

    Each time i read thing like ones wrote above i think Dr Hipps (SQLite author) should have choosen a zlib-like licence. How much could cost to put a credit or an acknowlodgement for SQLite? Nothing (well, some bytes in doc/pdf but few). SQLite is public-domain and it’s licence say nothing about credits, you can use it without them, but i’m morally forced to say “I use SQLite” but it’s my moral, not Adobe’s one (has one?… I’ll ask it to Noam Chomsky)

    Perhaps what Adobe wanted to do is make believe that no one needs to know they use external 3rd party opensource code, this can discredit the company. It’s how marketing guys thinks/works. But doing so and don’t saying nothing in documentation discredit Adobe.

  5. June 21st, 2007 at 9:58 am
    Chris Seahorn is reported to have said:

    I think Eduardo’s reply exemplifies the cheap shot. Even before Paul addressed the reasons the documentation may not be as morally or honorably written as you guys might expect, most users are fully aware of the choice to bring on SQLite and there are few (if any) publicly written articles right now on Adobe AIR that do not mention SQLite in tandem with the platform. It’s not as if Adobe is hiding the fact or trying to be clandestine about it and now after hearing Paul’s response I not only believe him, I’m sure their intention was not be slight SQLite. I’m sure they will now go out of their way to remedy the situation (after all…it is still beta and everything is subject to change).

    I also agree that I don’t think Adobe has to meet any expectations as compared to Google Gears in regards to SQLite changes or alterations. I really don’t think Gears is using an out of the box SQLite implementation either and since Adobe already had a round of this type of rhetoric with how they would change or alter Webkit and if it would be shared with the community (which it will be)…I don’t see Adobe having a bad record on sharing or exposing alterations or implementation based on opensource tech. I’m sure they will handle the SQLite community as graciously as they will the Webkit community.

    On confusion….if I am a Flex developer or an Ajax (etc) developer, how is choosing the correct SDK confusing? It’s not as if they have to be run simultaneously or you have to switch back and forth. Ajax coders have no need for the Flex implementation and Flex users the same in reverse. Your paragraph seems to indicate a jumbled mess and I don’t see it as such. I’m pretty sure devs will know which SDK applies to them and never have to worry the unchosen one. The fact Adobe put the extra effort into making the experience specific to the SDK chosen is a plus IMO..not a minus….and surely nothing resembling confusing. I don’t mean to make an enemy of you but by the time I finished your first paragraph, I looked back up to see if you were affiliated with M$ because I could swear it was conceived by someone with an agenda to dissuade users from AIR. Sorry if I called it wrong but with so many players in the game you are either friend (who may wish for things or want to suggest things) or foe (who looks for every opportunity to blacken an eye with even the silliest of reasons…some of which are unfounded but as long as they get snaked by search engines are effective).

    Chris

  6. June 21st, 2007 at 10:23 am
    Paul is reported to have said:

    @Eduardo:

    You’re absolutely right that it doesn’t cost any money, and only a trivial amount of file size and space, to acknowledge that AIR uses SQLite for its database engine.

    What I was hoping to communicate was that file size and money aren’t the only considerations that I and others had in mind when we considered the issue. Unfortunately, there is a real legal obligation that comes along with making something explicit in our documentation, different from mentioning it in a blog, discussion list, etc. Is the risk from that obligation (and other possible negatives) worth the downside of not giving acknowledgment to SQLite’s authors? That’s the tough question we had to face — ironically if the licensing terms were different we wouldn’t have had to make that choice =). So we did make a choice; perhaps it was the wrong one; fortunately we have an opportunity to reconsider our choice.

    As a side note, one of the reasons Adobe chose to use SQLite was because there is an opportunity to “give back” any improvements that might be made. If the licensing terms had been different for SQLite, I think it would have been much less likely that Adobe would have chosen SQLite (my opinion — I’m definitely not the one who makes those choices). Likewise, Adobe has stated that WebKit was chosen as the HTML engine in part because it’s an open source project and improvements can be contributed back.

    On top of that, we were always aware that, whether or not it was explicit in the documentation, word would get out that AIR’s database engine is based on SQLite. Personally I’m very glad that the word got out sooner rather than later — I’ve got tremendous respect for SQLite and the effort that’s gone into making it great, and I’m hoping that the benefit will work both ways — Adobe gets a great database runtime, and SQLite gains more public awareness and recognition since some big players (Google and Adobe) have chosen SQLite.

  7. June 21st, 2007 at 11:21 am
    Tim Anderson is reported to have said:

    Chris,

    If you read my post, you’ll see that I make it quite clear that Adobe is open about its use of SQLite - I even linked to the press release. My point was that SQLite is not credited in the documentation. Paul’s response above shows that this is an interesting issue, I don’t need to defend the point further.

    I don’t intend to be either friend or foe to a particular vendor, either one would be a failure.

    I still think the SDK download options are confusing, but I’ll happily agree to differ. That doesn’t mean any hidden agenda though.

    Tim

  8. June 21st, 2007 at 12:24 pm
    Chris Seahorn is reported to have said:

    Tim,

    Fair enough :)

    If I jumped to an incorrrect conclusion on your agenda, I sincerely apologize.

    On the SDK’s, I’m content that we agree to disagree. :)

    As for the future, I’m really interested in Paul’s latest article regarding encryption. Now this is something I would like to see become internally handled by AIR as opposed to an external class addition (much like I prefer an internally handled SQL database as opposed to a third party (Artemis) implementation) so it’s less confusing to new users and lets us become more of a viable choice (a choice mind you….not saying they should switch or would want to:)) for current users of commercial wrapper tools. I’ve really hinted to this for a long time in alpha and hope I’m happily surprised to see it come on board just like I was to see SQLite appear in beta (and where previous stage indications were it would not or was not part of the roadmap).

    Chris

  9. June 25th, 2007 at 2:20 pm
    Chris Seahorn is reported to have said:

    The “full text” search as part of the wish list has for some reason been nagging at me. While I would like to see it supported, I also think between the abilities offered with the AIR command set, AS3 command set and the commands we are able to use in our implementation of SQLite….I don’t see why it’s not possible to a least nearly emulate the ability of an actual fulltext sql search with some creative workarounds.

    Case in point…I wanted to add search to my AIR based offline forum just like my online forum has. That gets a bit tricky in this environment because I lose the benefit of PHP where crossing tables and nesting queries is a no brainer. Long story short I work out a routine to do just that. In my example I need to search all post bodies for a term. Since I display posts with an item renderer it’s stupid to simply display the result data from the post body search query because I will end up with a massive list of posts that are harder to correlate what forum or thread they belong to when a user chooses to reply (which can be done but visually they would have no idea what forum or thread they were responding too since all results are lumped as one). Instead I want to take the data from the post search query and re-use it for a nested search against thread titles and display the results via the parent thread (which it does) so I can grab the associative data (not shown to the user) when they normally choose a thread to display so my outbound post data is correctly inserted in the correct forum and correct thread depending on what they reply to when they sync their local board.

    Now….with this final result set from the two nested queries, I could just as easily filter how it’s displayed (the order… like a full text search would offer) using a sortOn method. For instance list it by name alphabetically and then by date (or whatever) based on data returned from both queries individually or in conjunction….no?

    I get a headache thinking about it but here are screenshots of the board and one of the nested query returning (it’s the first screen shown)

    http://www.flex-fanatic.com/index.php?cid=3&did=215

    displaying the final result and I wholly plan on seeing if my thoughts on this fly once I finish this project. I have a feeling that our being able to rely on AS3 may have some benefit for whatever is lacking in “our wish lists” :)

  10. June 26th, 2007 at 7:30 pm
    links for 2007-06-26 « thebadtiming is reported to have said:

    […] Why doesn’t Adobe’s AIR dev guide mention SQLite? A response to Tim Anderson, a bunch of Words, … (tags: sqlite adobe google gears air flex database) […]

  11. August 22nd, 2007 at 4:09 pm
    Jeff is reported to have said:

    I was reading the Adobe documentation, and thought to myself, “Hey this looks just like sqlite!” I then googled for sqlite and Adobe and found this page.

    Mentioning SQLite in Adobe’s documentation would give me a very positive and immediate impression of AIR.

    I would know to skip that whole section because it’s already rote in my mind, and AIR would seemlessly inherit the halo that hangs above SQLite.

  12. August 22nd, 2007 at 4:44 pm
    Paul is reported to have said:

    Hi Jeff,

    Thanks for the note and for sharing your experience — it really is very valuable for me to hear what things would help make the documentation (and the product) better for you.

    As I suggested in the post, I’ve made some changes in the documentation for the AIR local SQL database — it now says that it’s SQLite in the first paragraph (the first sentence I think), plus I added a section on some of the differences between the AIR implementation and the SQLite implementation as described in the SQLite docs.

    These changes will be included in the next public release of AIR.

    (I had to smile at the mental image of the “seamlessly inherited halo” — thanks for that, too =)

  13. February 25th, 2008 at 1:26 pm
    Tim Anderson says “AIR desktop platform tries to do too much for one product” « Flash Enabled Blog is reported to have said:

    Kramer auto Pingback[…] http://probertson.com/articles/2007/06/19/air-sql-docs-dont-mention-sqlite-my-response/ […]

  14. April 5th, 2008 at 5:27 am
    Al is reported to have said:

    Although the assumption that the majority of developers using AIR are web developers, as evidenced by another comment, there are those who are attracted to AIR primarily for it’s connections to sqlite, and/or the fact that there is an embedded database involved. So regardless of how one might feel about acknowledging the sqlite connection, the fact that it is involved is bound to garner the attention of what I would predict is a sizeable chunk of the development community who would otherwise not show an interest in the product. To the extent that it’s feasible, i think the needs of those developers (“under the hood/db-centric” as opposed to “web” developers) should be considered in the documentation. Thanks!

  15. June 2nd, 2008 at 7:28 pm
    chris seahorn is reported to have said:

    […] article! Reading Tim??s article his first paragraph would tend to make me believe he has an …http://probertson.com/articles/2007/06/19/air-sql-docs-dont-mention-sqlite-my-response/MySpace.com - Chris - 45 - Male - New York - http://www.myspace.com/chris …MySpace profile for chris with […]

  16. June 29th, 2008 at 5:44 am
    Tim Anderson’s ITWriting - Tech writing blog » Why doesn’t Adobe’s AIR dev guide mention SQLite? is reported to have said:

    Kramer auto Pingback[…] That’s the fuzzy summary version — for the fuzzy long version, see my post Why doesn’t Adobe’s AIR dev guide mention SQLite? A response to Tim Anderson. […]

  17. November 30th, 2008 at 6:02 pm
    Andrew Henze - A Qwentès Celebrity!: Adobe Air SQLite is reported to have said:

    Kramer auto Pingback[…] Adobe Air SQLite http://coenraets.org/blog/2007/10/new-air-sqlite-administration-app-with-source-code/http://www.peterelst.com/blog/2008/04/07/introduction-to-sqlite-in-adobe-air/http://probertson.com/articles/2007/06/19/air-sql-docs-dont-mention-sqlite-my-response/ Posted by Andrew Henze at 10:29 AM […]

Add your comment





Comment notes

Please keep comments on topic. Comments that are inappropriate or offensive will be edited or removed.

Paragraphs and line breaks are automatically converted to HTML, and quotation marks are converted to “smart” quotes.

The following XHTML tags can be used: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> . All others will be removed.

Articles by Type

Articles by Topic

Random Reading

Currently...

Subscribe