My application has thousands of records in one particular table that I need to populate the display with. I was wondering if I can implement paging to speed up the retrieval of those records?

In fact he specifically wanted to know if it was possible to do data paging with the SQLRunner class in my air-sqlite library. The answer is yes it works, without even needing any changes to the library. See below for how to do that.

The easiest way to implement data paging (in other words, getting only a subset of a query’s results at a time) in a SELECT statement is to use the LIMIT..OFFSET clause in the SELECT statement.

In summary, you can put a LIMIT clause at the end of a SELECT statement and the result set will only include the specified number of rows:

SELECT * FROM myTable
LIMIT 3

Then to get the next 3 rows, you add an OFFSET value:

SELECT * FROM myTable
LIMIT 3 OFFSET 3

Fortunately, you can use statement parameters for the LIMIT and OFFSET values (although the docs don’t mention this — shame on me!):

SELECT * FROM myTable
LIMIT :limit OFFSET :offset

Then you just specify values for those parameters and you’re good to go.

I tested this using the SQLRunner class by adding a new set of unit tests and it worked just fine without any changes to the library. (Hooray for unit tests and FlexUnit support in Flash Builder 4 — they made it nice and easy to test this out since I already had the infrastructure in place.)

Here is the SQL statement I used for the unit test using LIMIT..OFFSET with parameters:

SELECT colString,
colInt
FROM main.testTable
LIMIT :limit OFFSET :offset

And here is the line of code that calls the statement (modified slightly for readability). test_result is the result handler function. The parameters object specifies that I want 7 result rows, starting with row number 3 (i.e. skipping 2):

_sqlRunner.execute(SQL, {limit:7, offset:2}, test_result);

As Peter points out in the comments, another approach to do something similar to data paging with AIR is to specify a number of rows to retrieve as the first argument to the SQLStatement.execute() method, then call the SQLStatement.next() method to retrieve additional rows from the same statement. This technique is described in the documentation so I won’t go into more detail about it here except to say that it does have a couple of drawbacks (mentioned in my comment below) that make it less suitable for data paging (but still very useful).

Warning! To add the missing feature I had to introduce a non-backwards-compatible api change. Read the details in the project history.

So, the bug is fixed, the feature is added, the version number is incremented (0.1.1 beta), and the code and SWC are live on Github.

Download the version 0.1.1 SWC

Read about the changes

In conjunction with the release I also added a couple of new examples to the project page, including a “bare bones” code only example for quick starters, and links to an example of using the library in a Robotlegs application (the example app from the 360|Flex training).

Enjoy, and as always feel free to report problems as issues in Github or in the comments on the project page.

For lack of a better name, I call it my “AIR SQLite utility library”

The code currently contains one major piece of functionality (well, two different variations on one bit of functionality), which is a SQL “query runner” library. This is a wrapper for the AIR SQL classes that allows you to run a SQL statement by just passing a few bits of information:

• The text of the SQL statement itself
• An object containing properties with the values for any statement parameters in the SQL
• A Function to call when the operation completes
• A failure Function
• (optionally) a class to use as the data type for the data returned from a SELECT statement

The library does all the work of creating SQLStatement objects and caching prepared queries, as well as pooling SQLConnection instances so you can execute multiple statements simultaneously. It also has a variation that allows you to specify a “batch” of statements to execute, and they are executed in order in a transaction.

I’ve also got an additional utility to add to the library. It’s a “database copy” utility that allows you to create a “deep copy” of a database — all it’s tables, views, etc. — with or without data. The key reason why this is useful is that you can use it to create an encrypted database from an unencrypted database (and vice-versa). It’s written and tested, but I decided to modify the structure slightly before releasing it, so it’s not checked in yet.

I’ve put the details about how it works and why it’s designed that way in the project page. In case you’ve ever wondered how I design apps, I think the examples and this library give some insight into how I actually do my database-driven AIR app development. At least, how I structure the data-access part of my apps.

On (another) personal note, this project is also my first project that I’ve posted to my Github repository (as opposed to projects I’ve forked). It was actually posted-but-not-advertised on Google code for a month or so, but I decided to move to Github because the collaboration and checkin-without-network-connection features are so awesome.

A couple of years ago, before AIR beta 1 (“Apollo”), I wrote a tool for testing SQL statements in AIR. Later (I can’t remember exactly when) I released it under the name “Doppler.” I had plans to make it a full-fledged database management tool, but those plans never really panned out. I’ve told this story lots of times now, about how I now use (and even help develop) David Deraedt’s “Lita” SQLite admin tool.

However, “Doppler” in its current form served a different need than Lita — Lita is great for creating and managing database structure (you’ll never have to write a CREATE TABLE statement again), but it isn’t as strong for testing individual SQL statements. Over the past several months I’ve been doing some “serious” app development, working on an AIR app that makes heavy use of SQLite. I found my little tool to be quite handy, and also found and implemented a number of small improvements that make it much more useful.

Since my original intentions were never going to come to fruition, and I had new future plans for the app, I decided a new name was in order. Since it is for running SQL statements, “run” was an easy and obvious choice. So now, (several weeks after posting it), allow me to officially introduce you to the Run! AIR SQL query authoring and testing tool — the next generation of Doppler.

If you look at the full feature list for the latest release, you’ll see that I’ve added several types of changes. A few of the ones that I’ve found most useful in my day-to-day work are:

• Auto-update and badge install — no more downloading the .air file (admittedly not a big deal) and no more uninstall and reinstall each time there’s a new version (a big deal)
• “Recent SQL files” and “Recent databases” menu items
• Opening and saving SQL files (plus keyboard shortcuts)
• “Row count” and “execution time” displayed for SELECT statements
• Line numbers and auto-indent in the SQL panel

And yes, I did say “day-to-day” work. On some days, Run! has been my primary IDE, as I’m writing and testing and rewriting large SQL statements. It’s been kind of strange and kind of fun building a developer tool and using it as much as I use Run!.

I’ve already gotten a bug report and I’m working on features for the next big release, so if you have thoughts or comments please comment on the project page!

As David pointed out, the fact that I’m an Adobe employee and am participating in the project doesn’t mean it’s now an official Adobe product (for good or bad). I’m doing this 100% on my own time. And my role is still pretty small — David is certainly the lead, main, primary, controlling, etc. author. I just file bugs as I find them, add some comments about feature requests, and fix issues when I know how to.

In case you’re curious, the back story is really pretty straightforward. I started writing my own version of an AIR SQLite admin tool back before AIR beta 1, but never had time to take it beyond a “query runner” tool. Late in 2008 I discovered Lita, and once I started using it I realized that 1) it is implemented in a similar way to many of the ideas I had, and 2) It’s already got a big head start in features, so there’s not much point in me trying to “compete” or anything like that, especially for something that I wasn’t planning to make money from.

After using it for a while I discovered a few bugs, and decided to email David about them. (He and I had communicated a bit previously, about the encrypted database functionality added in Adobe AIR 1.5 and how to integrate it into Lita.) I think I probably mentioned in that message that I would be willing/interested to fix issues myself as I have time. Fortunately David was very kind and accepted my offer. So, as I said, now when I find issues I get to fix them myself, which is nice because I can fix them quickly, but also adds some responsibility since now the burden is on me to make those changes myself =)

As a side note, I really want to point out that David is a really great developer — something I appreciate greatly as we work in the same codebase. It is a big testament to his architectural and coding skills that I was able to dive right in and fix four bugs/feature requests in a very short time (literally a matter of minutes after first looking at the code). I’ve learned a lot just from seeing his code, and now I’m anxious to read his “Flex Architecture Fundamentals” series to learn more about the thinking behind the great code.

But not any more. A few days ago I checked in a beta (barely tested) version of that adds support for Flash Player 10 (browser) projects. It’s not linked from the main google code project page yet, but you can get to it from the GZIP encoder project downloads page.

The download that adds Flash Player 10 support is “ascompress-.2.0.zip”. It now includes two SWC files:

• GZIPEncoder_AIR.swc is the full package, with all the same features as the previous release (.1.0.2).

• GZIPEncoder_Flash.swc is the Flash-Player-only version. It only compresses/uncompresses ByteArrays, and doesn’t have any support for files.

  Of course, since Flash Player 10 also adds local file opening/saving support that uses ByteArray data for the files, you could easily write your own code to prompt the user for a gzip file, or compress a file and prompt the user to save it. For an example of opening and saving files from Flash Player 10, see my quick start article: “”

As I implied when I described this as “beta (barely tested)” I haven’t done a lot of testing of this code yet. I don’t actually anticipate that there will be issues — really all I did was refactor the code into two classes instead of one — but that doesn’t mean there won’t be any issues, of course. =) If you’re interested, and especially if you’re one of the people who’s asked for this support, I would love it if you try it out and let me know how it works for you. If you’re willing to test it out, I’ll be happy to quickly make any fixes necessary.

One thing I’ve been wishing for the last week or so is undo/redo functionality as I’m editing query text. Coincidentally, I just found out that my 360|Flex friend Jac Wright has written a library for undo and redo in Flash Player (ActionScript) text fields. The previous link is to the Google Code project; here’s his introductory blog post about the “undo textfields” library. (via Tyler Wright via Gilles Guillemin/Twitter)

Admittedly I haven’t tried this yet, and I’ve asked whether it’s been tested in AIR so I don’t know whether it will actually be feasible as-is. But I’ve got my fingers crossed! =)

As with previous versions, if you’ve been using the app you’ll need to uninstall it before installing the new version. Someday I’ll add updating support, but I’m not going to promise anything real soon.

Along with this release, I’ve also made a change to how I describe the tool, and to my future intentions for it. I’ve always had it in my mind to make this a full-fledged database admin tool, similar to the tools that come with SQL Server or other enterprise databases. However, time has obviously not allowed me to do that, and in the mean time other tools have been released by other developers. I’ve found one, David Deraedt’s “Lita” SQLite admin tool that is sufficiently mature that I use it in my day-to-day work now and it definitely beats doing things by hand! There are still improvements to be made and features to be added, but when I’ve reported bugs and feature requests he’s been quick to respond and release updates.

So while I’m sure nobody’s been holding their breath waiting for me to finish the “admin tool” portion of my app, I just wanted to clarify the new direction I’m taking it — or rather, the fact that I’m not planning to take it in as many new directions! (Hence the change in title for the project from “AIR SQL admin tool” to “AIR SQL query testing tool.”)

That doesn’t mean I’m done developing this tool by any means. In past jobs where I did heavy database development, and in a project I’m currently working on that involves heavy database development, I find it very useful to have two different kinds of database tools — one for creating and managing database objects and structure, and another for testing queries. While Lita does in fact have a tab for testing queries, I personally find Doppler to be a bit (not a lot, but a bit =) more developed in that specific area. On the other hand, Lita certainly does a lot in the db management space that Doppler doesn’t do. So I find the tools very complementary in terms of my actual development work.

As always, I welcome feedback, questions, thoughts, etc. And thanks to everyone who’s already reported bugs and offered suggestions!

As you may have heard among all the news that’s coming out from MAX right now, Adobe AIR 1.5 has just been released. Most of the features of AIR 1.5 are features that were introduced with Flash Player 10. However, one new feature in AIR 1.5 that helps address the easy-to-read database issue is AIR 1.5’s new support for AES encrypted databases.

I’ve had the interesting and at times complex job of writing the documentation for this new feature. The bulk of the new documentation is in a new section “Using encryption with SQL databases” in the SQL database chapter of the manual “Developing Adobe AIR Applications”.

It’s pretty straightforward to use an encrypted database. You have to create the database as an encrypted db. To create or open it, you call the SQLConnection class’s open() or openAsync() methods, just as you normally do, and there’s a new parameter where you provide a 16-byte ByteArray encryption key for the database. That’s all there is to it.

The SQLConnection class also has a new reencrypt() method, that you use to change the encryption key of an already encrypted database.

If you want to see some code examples, check out the documentation listed above, or see these quick start articles on the Adobe developer center:

• Working with the encrypted local SQLite database (Flex)
• Working with the encrypted local SQLite database (Flash)
• Working with the encrypted local SQLite database (HTML/JavaScript)

There are a couple of parts of the new documentation that I think are particularly interesting (although I’m obviously biased since I wrote it all):

• Considerations for using encryption with a database talks about some of the various reasons you may want to use an encrypted database, and some of the differences in how you might architect your application to account for the desired level of security and privacy.

• Example: Generating and using an encryption key goes into great depth into one technique for creating an encryption key for your database. You have to provide a ByteArray encryption key to encrypt your database. How you come up with that encryption key can have a big impact on the actual security of your app data.

  This was definitely the most involved example I’ve written for the documentation. the techniques it uses were specified to me by security engineers at Adobe. The documentation goes into lots of detail describing how these techniques are used and why. Both the code and the text went through a series of reviews by engineers and security experts.

  I ended up writing the example as a simple UI, plus a class that anyone can just pull into their own app (rather than having to pull apart the example and turn it into something you can use. I’m also excited to share that Mike Chambers and Christian Cantrell decided that this “encryption key generator” class is useful enough that it’s now included in the open-source ActionScript 3.0 core library (as3corelib) project.

Now that AIR 1.5 is out the door, I’ve updated my AIR database admin tool to support encrypted databases (when you try to open an encrypted database it prompts you for an encryption key, which you enter as a Base-64 string). I already mentioned that I wrote the encryption key generator class that’s now in as3corelib. I’ve also got at least one more new encrypted database-related open-source project that I’m about to release…but I’ll wait until the MAX “firehose” dies down a bit before I write more about that one. =)

As I mentioned briefly before, this week I presented at the 360|Flex San Jose (August 2008) conference. My presentation was titled “Adobe AIR SQLite: An optimization conversation.” As I mentioned in the presentation, the term “optimization” could mean a few different things — for example, optimization meaning improving performance, or optimization meaning improving developer productivity. While my presentation focused mostly on the first type of optimization, I included suggestions for tools, libraries, and strategies that fall in the “developer productivity” type of optimization as well.

Anyway, as always I’m happy to make my presentation materials available. Here are the slides (with some notes) from my presentation:

“Adobe AIR SQLite: An optimization conversation” slides (PDF in .zip - 504kb)

I don’t really have any specific code examples, apart from what’s in the slides, so there’s no “source code” download. However, I did link to a lot of external tools and resources (including a few of my own). To save you the trouble of digging into the PDF, here are the links:

Tools

• Cristophe Coenraets’ “SQLite Admin”
• My tool for testing queries
• “Lita” by David Deraedt (I learned about this one after the presentation, so it’s not discussed in my slides/video, but I thought it’d be worth mentioning anyway.)

Application architecture/patterns/libraries

• SQLite MXML wrapper classes (Peter Elst)
• Data access layer (Brandon Ellis)
• asqlib SQL statement generator (Miran Loncaric)
• “Command” classes (me) as demonstrated by my AddressBook sample application
• AIR ActiveRecord source and blog post explaining its usage (Jacob Wright)
• AIR SQLite Cairngorm services (Eric Feminella)
• Connection and statement pools, mentioned (with source code) in the article User experience considerations with SQLite operations (Daniel Rinehart)

Finally, as you may have heard, Adobe sponsored the recording of every presentation at 360|Flex, and they’re all going to be made available free of charge via a channel in Adobe Media Player. They’re rolling them out in phases, and mine isn’t available yet. When it is, I’ll update this post with the video as well. Update: the video is now available on Ted Patrick’s blog as well as in Adobe Media Player.

In the mean time, 360|Flex was full of awesome presentations. I wasn’t able to get to all the ones I wanted to see, due to conflicts and me trying to finish up preparation for my presentation. So I’m going to be spending some time watching many of those videos as well. If you’d like to see the videos, Ted Patrick has posted instructions on his blog:

How to view 360|Flex San Jose 8/08 session videos in Adobe Media Player

Having been down this road before with other languages, I decided to put together a simple tool for testing out regular expressions. I wanted to be able to enter a pattern and a test string, and have it check to see if it was a match. That way, I could test out various possible good matches (and non-matches) to see if they give the right result. I was using named capturing groups quite a bit on this project, so I added an extra feature where it shows all the groups that are captured as well.

Anyway, I realized this might be useful for anyone who’s trying to use regular expressions in an ActionScript 3.0 project, so I thought I’d make it available.

Want to test it out? Visit the project page to see the tool and download the source code.

If you have comments or improvements for the project, feel free to add a note in the comments on the project page.

The library is still in ActionScript 2.0—I haven’t finished porting it to ActionScript 3.0 yet, although I’ve started.

This release (.3 beta) contains a single bug fix. In anticipation of porting the library over to ActionScript 3.0, I wrote a series of unit tests, and—what do you know—I found a bug in the code that parses date values returned from an XML-RPC service call. I guess unit tests really are useful for ActionScript =).

I also added a new project history page and a new step-by-step example of how to make an XML-RPC call, in response to some user feedback and questions.