It turns out that my thinking on that topic isn’t going to become anything in the Adobe documentation. So I’ve sort of just been sitting on my ideas with the idea that someday I’ll probably share them.

Which brings me to this post.

Aside: upcoming presentations on multi-screen development

This is really a side note to the main ideas here, but I thought I’d mention it since it’s the reason I actually stopped to write this. In one of my mind-wandering moments this morning, I realized that at the upcoming 360|Flex conference (San Jose, March 7-10) I’ll be involved in two presentations that are directly related to practical aspects of building multi-screen apps:

• As part of the pre-conference free training, Joel Hooks and I are doing a four-hour training on using the Robotlegs micro-architecture. (Joel is the main speaker, fortunately, because he’s one of the core Robotlegs developers and the main “evangelist.” I’ll be helping out, to try and help save his voice and maybe to help find typos. =)

  If you’re wondering what a Robotlegs session has to do with multi-screen apps…keep reading, it’s explained below.

• On Tuesday and Wednesday, Renaun Erickson and I are giving (two separate) presentations about building iPhone apps using Flash Platform tools. My session is more of an intro into the workflow, how it’s similar to other app building, and how it’s different. (Plenty of code to look at — don’t worry!) Renaun is going to get more into specifics of building a game, as well as some more intermediate/advanced topics like performance.

Funny that I never really made that connection until this morning. I guess subconsciously I knew this is a direction I want to move in, and that’s why I’ve been focusing my learning (and hence my presenting) on these topics =)

Background

While I was partially trying to keep my thinking abstract, from a practical perspective I centered my thoughts around developing using the Flash Platform. So conceptually, what I was thinking about was the idea of making a single “app” for multiple screens, which would mean that (in the technology of the next year or so) it could be potentially take several forms:

• a browser/computer app (Flash Player)
• a browser/mobile app (Flash Player)
• a desktop app (AIR)
• a mobile (iPhone) app (Publisher for iPhone, similar to AIR)
• a mobile app for other devices (AIR)
• a TV app or widget (Flash for the digital home)

(Obviously many of these are only out in public beta, some have only been vaguely publicly acknowledged, and others may or may not ever exist as actual Adobe products =)

One app, or many?

Back when I started thinking about this, there seemed to be two main camps. One group was a strong advocate for the “single source” idea: you would build one SWF or one AIR app and distribute that same app on any platforms (desktop/mobile/TV). Obviously a key element to that, especially for desktop/mobile apps rather than browser apps, would be having code that detects screen size and device capabilities and adapts the UI to the device.

The second group felt that it was more likely that developers who are creating an app for multiple screens would actually create multiple apps (from the perspective of the IDE). The apps would obviously share some code, visual assets, etc., but would be different enough that they’d be created separately and distributed as separate file types. (In the case of apps created for the iPhone, of necessity this has to be true, at least in terms of the publishing part.)

Since I try to be pragmatic about things unless I have a really good reason, I favored the second viewpoint. However, I acknowledge that in some circumstances the first approach might be used. As always, I think that developers are going to use a mix of approaches.

For example, suppose I’m building an app and I want to create a version for desktop computers, a version for the iPhone, and a version for other mobile devices. Personally I see this not just as an issue of “porting” the app from one platform to another. I think that each device has its strengths, and more importantly, each device is used for different purposes and in different contexts. If I’m building an app for a device, I should definitely be taking that notion of context into account as I’m designing the app. That pretty much rules out the possibility of using the exact same app (UI and everything) for both computers and mobile devices, unless I want it to run as a widget-type app on the desktop.

Also see: Adobe Flash Platform contextual applications developer center

Adaptable code to the rescue

On the other hand, that doesn’t mean that developers will always create completely separate projects. For example, suppose I’m creating an app for iPhone, and another for another mobile device. Or perhaps I’m using the Publisher for iPhone to create an iPhone version of my app and an iPad version. If my app is a game where I can more freely discard the conventions of the platforms, perhaps the only platform difference I need to consider is the difference in screen dimensions and pixel density. In that case, it’s quite possible that I can .

Even for a more line-of-business or productivity app, one of the key ideas in Slider, the future Flex mobile framework, is to (as much as possible) abstract away platform convention differences. For example, if iPhone usually puts the back button on the top, and another platform puts it on the bottom, and another platform puts it in a menu, and another platform has it assigned to a physical button on the device, then Slider might have a “back” event that you can hook into, and you can make your app perform the necessary “back” tasks regardless of platform.

Design patterns to the rescue

And, of course, in some situations you’ll almost surely need to create different versions of apps for different screens. For example, suppose you’re building a desktop Twitter client and a mobile Twitter client. Chances are good you’ll put some functionality into the desktop version that doesn’t go into the mobile one — such as previewing images, or maybe Facebook integration. On the other hand, adding a feature like automatic location tagging would make lots of sense in the mobile version, but not so much in the desktop one.

As I was thinking, many months ago, about the idea of efficiently creating different versions of the same app for different devices, a thought hit me like a ton of bricks. This is exactly the use case that the oft-mentioned “separation of concerns” is designed for:

Over several years, various “micro architectures” and “application frameworks” have emerged and waxed and waned in popularity. Many of these architectures are modeled around the “Model-View-Controller” design pattern (or its many variations).

One of the key benefits that these architectures claim is that it helps you keep the pieces of your application separate, without explicit links and dependencies between them. I’ve looked into and even tried out several of these over the years. One example that is frequently used to describe the notion of separation of concerns, which I always struggled with, goes something like this:

Suppose you’re creating an app, and you build the user interface and the other logic like server communication and data processing. If you use MVC/separation of concerns, then it’s really easy to just rip out your whole user interface layer and replace it with a new one.

They usually lost me with that one. As much as I tried, I couldn’t imagine a situation in which I’d want to build my app and then just rip out the UI and replace it with a different one note 1.

Until now.

Suddenly I had discovered, for myself at least, a real-world use case for the separation-of-concerns-so-you-can-swap-out-the-UI argument. If I’m building an app for the desktop and mobile, I’d like to be able to reuse my code where I can. At the same time, some functionality is only going to apply to one app or the other, so it’d be nice to be able to plug it in cleanly. Oh and by the way, the user interfaces are going to be different, so being able to swap those out is an absolute must.

I’d always liked micro-architectures in general, if only because I like knowing that I’m building my app using some pattern or structure that is based on developers’ real-world experience. It’s much nicer than trying to invent it myself and having to deal with all the pain points they’ve already gotten over. Now that I am imagining a world where I create multiple versions of the same app, with similar functionality but different user interfaces, suddenly micro-architecture patterns have become indispensable in my mind.

Conclusion

These are only just a few of my thoughts about the future world of multi-screen, contextual applications. Like Lee Brimelow, I believe that going forward building multi-screen, multi-context apps is going to be a much more common scenario. From my perspective as someone who thinks a lot about user experience design, and trying to optimize tools for the task and context, I think this is one of the most exciting aspects of the current technology revolution. I’ll definitely continue to share my thoughts and ideas in the future (hopefully more practical ones, too, not just abstract ones like this =)

Notes

Note 1: Just to be clear, I’m not trying to sound negative about MVC or micro-architectures/frameworks. Most of them provide many benefits, many of which are also related to the idea of separation of concerns, such as making code easier to test, making code cleaner, reducing boilerplate code, providing structure so that teams or developers who inherit a project can get going more quickly, etc. Swapping out the view layer is just one little benefit they mention. (back)

We’re mainly looking for developers who have Flex experience but little or no experience developing for Adobe AIR. There aren’t many other restrictions — We’ll conduct the study on the phone and online using Adobe Connect.

Admittedly, I realize that if you read what I write here then there’s a good chance that you’ve already got too much AIR development experience. Even so, we’d appreciate it if you can spread the word to other developers you know who might be qualified.

If you’re interested or want to get more details, check out the official post on my team’s blog:

Need participants for studies about AIR and Flex

On a related note, we’re also conducting some (very brief) surveys about your experience developing AIR applications (Flex or HTML/JS). I can’t remember all the places where you might encounter them, but if you browse around the documentation or the developer center for a while there’s a chance you’ll be offered the survey. If you’ve done some AIR development and get a chance to take the survey, we’d like to hear about your experiences.

I missed the initial announcements and only became aware of Prism a couple of weeks ago when I happened on a discussion of Prism by Mike Chambers (complete with rather emotional comments on many sides). I can’t say I agree with everything Mike says there — it sounds like at the time he wrote it he didn’t have a complete understanding of what Prism was/is — although certainly some of the people who commented obviously didn’t have a great understanding of Adobe AIR. In any case, it’s an old (by Internet time) post, so I’m sure nobody in that conversation would want to be held to their stated opinion.

My opinion about Prism is more in line with something JD shared recently, specifically the quote below which I think clarifies the value proposition (and relative merits) of both AIR and Prism:

For me, the top difference between the Adobe Integrated Runtime and what I currently understand of Mozilla Prism is the balance between creator choice and user choice. AIR lets you create a predictable beyond-the-browser experience; Prism lets the developer indicate how they’d like the presentation to appear, but the enduser can still modify the markup, scripts and styles they choose to package up in Prism. Two different types of contract between creator and consumer.

Any existing webpage can be repackaged and modified in Prism; any web developer can create desktop-optimized experiences in AIR.

I think this is very relevant to one example that I’ve given to explain one of the benefits of AIR. There are now apps that people use in a browser (e.g. GMail), but they use them like a desktop application — opening them up and keeping them open all day. The user doesn’t want that browser window to navigate to a different page, and if some web site causes the browser to hang or crash, the user’s email app gets hung too. For an app like that, it makes sense for the developer to create a version in AIR that can be run as a separate application. However, there’s a big dependency from the user perspective — the app developer has to actually create the AIR app. (e.g. Google would have to make an AIR version of GMail). Once the developer decides to build an AIR app, there are other benefits and capabilities that become available — but the point is it’s still in the developer’s hands to actually build the app. The value I see for Prism is that with Prism the end user can choose to make that app a Prism app without the need for any action by the app developer. Of course, the app will still have dependencies on things like an Internet connection, and won’t have any more capabilities than any other browser-sandbox app. But it’s certainly no worse than what the user already has in the browser, with some nice conveniences added in.

Keith Peters wrote a similar post on the same topic, which I also recommend.

(via Jesse Warden)

I got an email from “Brent” asking me if it’s possible to give someone a url that links directly to a particular keyframe of a Flash application:

Maybe you can tell me if this is possible or not, I have a feeling it’s not cause i’ve been searching through google and not finding anything written about it. Can you target a specific keyframe in a flash movie through an HTML page link (possibly using PHP? or javascript to feed the info to flash?) The reason for doing this would be if, for instance you had a flash site with different sections and wanted to give someone a link to your contact section without having them navigate to it…

is this possible?

Indeed, it is possible.

There are a few ways to accomplish what Brent was asking about. In fancy buzzword circles they might call it “using REST with Flash” or “creating RESTful urls that work with Flash”, or something like that (the key term being “REST”, which means having distinct urls for direct-linking to different states of a web-based application).

I. Using Flash’s frame anchors

The first way is one you can do within Flash itself, without needing any ActionScript. You can assign “anchors” to keyframes on the timeline. If you’re familiar with adding frame labels to frames, it works pretty much the same way:

1. Select the keyframe you want to be able to target
2. In the Property inspector enter the anchor text (the text that will be added to the url to link to the particular frame) in the “<Frame Label>” field
3. In the “Label type” drop-down choose “Anchor”
4. Finally, to make it work you have to open File > Publish Settings, and in the HTML tab, in the Template field, choose “Flash with Named Anchors”

To link directly to a particular keyframe, just use this format: “html_page_name.html#anchor_name” (replacing “anchor_name” with whatever name you gave the keyframe in step 2 above).

Admittedly I’ve never used this in production. I did a quick test and it seemed to work fine in Firefox 2.0 and Internet Explorer 6; but I couldn’t say if it will work in other browsers.

Also note that if you want to use frame labels (e.g. for ActionScript navigation) and anchors on the same frame #, you need to create them separately. In other words, you need a keyframe on one layer at that frame with the frame label, and another keyframe on another layer of the same frame with the anchor.

II. Using FlashVars and server-side code (url variables)

For a more involved (but well-tested) approach, you could use FlashVars with PHP to pass in the url. Basically you would make your html page url look something like this:

html_page.php?section=profile

The value that you pass in to the “section” variable could be a frame label, or just a value that you’re going to use in ActionScript somehow.

Then in the actual php source, in the part of the HTML that embeds Flash Player, you’d put something like this in:

<object ...>
<param name="FlashVars" value="section=<?php echo($_GET['section']) ?>" />
...
<embed ... flashvars="section=<?php echo($_GET['section']) ?>" />
</object>

(Note, I’m just writing this php from memory, and it’s been a while since I did php, so I may be putting the wrong thing there — but hopefully you get the idea).

If you’re using one of the JavaScript libraries for embedding the SWF, such as SWFObject or Adobe’s Active Content technique, those libraries already have built-in functionality for FlashVars, so you should be able to find examples of how you’d add the FlashVars with those libraries.

Anyway, that’s the HTML/PHP side of things. Back in Flash, on Frame 1 of the main timeline you’d add code like this:

this.gotoAndStop(section);

The variable named section is ‘magically’ created by Flash Player, as a result of the FlashVars code that was written into the html by php.

For more on FlashVars, you can see this post I wrote a while ago:

Using URL Variables and FlashVars

And you can see these slides/notes from a presentation I gave to an Adobe users’ group covering FlashVars and related techniques:

Slides and notes from “Flash to external data communication”

III. Using Kevin Lynch’s technique for making linkable Flash applications

Finally, there’s an even more complex but fancy and re-usable approach that was shared by Kevin Lynch, formerly of Macromedia and now Chief Software Architect for Adobe. It’s more involved code-wise, but it’s pretty slick:

Making Rich Internet Apps Web-Friendly

This technique (like the first one) doesn’t require anything special on the server — it uses JavaScript and ActionScript to do all the work. The advantage it has is that it updates the url in the browser window as you navigate around in the Flash SWF — so that not only can you have urls that link to different application states, but it’s easy and obvious to discover what those urls are.

This is a good, if not comprehensive, list to think about.

Many of these seem to come from his experience with desktop app. design, and a lot of them were pretty familiar to me from HCI design courses I took in grad school (which is one reason why I usually did my class projects in Flash rather than in HTML, like so many others did — the range of interface/interaction design possibilities is so much greater in an RIA platform).

Be sure to watch the video!

Project page for DataTiles: A Modular Platform for Mixed Physical and Graphical Interactions

It is a project being led by Jun Rekimoto, Director of the Interaction Laboratory at Sony Computer Science Laboratories

(via Eric Dolecki)

http://www.sitepoint.com/blog-post-view.php?id=216588