1987 – Knowledge Navigator

During the John Sculley years, Apple loved a good concept video. The company produced three featuring the Knowledge Navigator, a device it considered the future of computing. Much like a concept car, nothing like it was ever produced.

The Knowledge Navigator was a large flatscreen that folded shut like a book. It featured video conferencing, access to networked data and automatic translation. Most importantly, it had an interactive assistant that looked like a human butler.

The assistant was part of the agent craze that was in vogue at the time. It wasn’t enough that a program managed information, it needed to have an onscreen persona. This thinking led to the Office assistant, perhaps the most derided feature Microsoft ever created.

1987 – GO Corporation

GO was founded to create portable computers and an operating system for running them. They pioneered a lot of work in the area of pen-based computing, but never saw any significant sales.

GO created the PenPoint OS, one of the first geared entirely for pen use. It had many breakthrough features, like using gestures for commands throughout the system. GO would later sue Microsoft for patent violations.

1989 – GRiDPad

Things got a whole lot more real with the release of the GRiDPad, arguably the first true pad computer. It was a portable device with a 10-inch stylus-operated display and a 386SL processor which enabled it to run DOS-based software.

Information about the GRiDPad is scarce, but it appears it was used by Chrysler for inventory management. Ruggedized versions were also sold to the U.S. military.

1963 – Sketchpad

Sketchpad was a revolutionary early CAD program that allowed the user to draw designs right on the screen using a pen. A user could, for example, sketch out machine parts and then combine multiple instances of them in a larger drawing.

Sketchpad used an input device called a light pen, which the user points at a CRT display. A sensor at the tip of the pen detects when it’s hit by the display’s electron beam. Software compares the time of the hit to the display’s scan timing and works out where the pen was pointed.

In spite of the “pad” name, Sketchpad was hardly portable: it required a large mainframe and a heavy CRT display. Nevertheless, it was a key inspiration for graphic user interfaces and object-oriented computing.

1972 – PLATO IV

Student using a PLATO IV terminal. Image from the University of Illinois.

PLATO was a series of educational computer terminals that originated from the University of Illinois. In the 1960’s and 1970’s, PLATO contained many features that we take for granted today like e-mail, message boards and online tests.

The fourth generation PLATO IV terminal featured a flat (and bright orange) plasma screen that students could touch to answer questions. The touch function was achieved by a series of infrared lights and receptors around the rim of the display. A finger would break a beam of light and trigger a touch.

In the 1980’s and 1990’s personal computers took over the classroom. These days, software that’s remotely descended from the original project is sold for PCs under the PLATO name. There’s also a community of PLATO users running the original software on a simulated mainframe.

1983 – HP-150

The HP-150 by Hewlett Packard was one of the world’s first commercial touch screen computers. It was an MS-DOS compatible device that looked much like a typical PC of the day, except with a nine-inch CRT touchscreen.

As with the PLATO, the HP-150 used infrared for the touch effect. The sensor resolution was quite coarse, so you could use it to select options but not for example to draw. Small sensors at the bottom of the screen would also malfunction when dusty, requiring the user to vacuum quite often.

HP predicted that developers would easily update their applications for touch use. Still, the HP-150 was not a huge success, perhaps because working with your arms raised is just not comfortable. A mark II model was released in 1985, but the series was quietly discontinued in 1989.

Coming in part 3: Finally mobile!

1966 – Star Trek

Star Trek introduced a world with gleaming ships, primary-colored uniforms and not a scrap of paper in sight. What Captain Kirk had instead was the PADD, a Personal Access Display Device for the 23rd century.

PADDs were remarkably close to the modern pad concept: a flat computer screen embedded in a frame the size of a clipboard. As the show was made in the 1960’s, they were also equipped with blinkenlights and an enormous angular stylus.

In later incarnations of the series, the PADD grew thinner and lost the stylus. While the Original Series never showed an actual user interface for the PADD, series from The Next Generation on used LCARS, a fictional OS. In the real world, the look of LCARS was designed by Michael Okuda.

1968 – 2001: A Space Odyssey

Arthur C. Clarke’s written version of 2001 featured the Newspad, a flatscreen news viewing device the size of a sheet of paper. A user could view a summary of headlines or zoom into a story by punching in its code – presumably by some kind of a keypad.

Switching to the display unit’s short-term memory, he would hold the front page while he quickly searched the headlines and noted the items that interested him.

Each had its own two-digit reference; when he punched that, the postage-stamp-sized rectangle would expand until it neatly filled the screen and he could read it with comfort.

In Clarke’s future, the word “newspaper” remained even though the concept of delivering news on paper had become outmoded. It appears he was off by a decade or so.

The movie version showed the Newspads being used as flat screen portable televisions. The first attempts to create an actual flat TV were not made until the 1970’s.

1972 – Dynabook

In the late 1960’s, researcher Alan Kay dreamed of a personal computer he named the Dynabook. He detailed his ideas in the 1972 paper called A Personal Computer for Children of All Ages, which was published while he worked at Xerox Parc.

The Dynabook was to be portable and flat, with a screen fit for interactive graphics or electronic books. While the illustrations of the paper show a physical keypad, Kay also hypothesized that touchscreen keys were a viable option.

Kay intended the Dynabook as an educational environment where users would write their own programs. This theory of learning was very influential on the much later OLPC project.

Coming in part 2: Early pen and touch interfaces

The good news is that it can be done with contributed modules. The bad news is you’ll need quite a bunch of them.

What you’ll need

Drupal 6

ImageCache

For creating presets and storing edited images.

ImageCache Profiles

For using ImageCache with profile images.

ImageAPI

Behind-the-scenes module for handling image editing.

Transliteration

Cleans up unusual characters from filenames.

Either the GD2 or ImageMagick library

Note: These are not Drupal modules, and their installation instructions are too complex for this tutorial. GD2 is already installed on many servers and web hosting services.

Step 1: Check url and download settings

You’ll need to enable either the clean URL or private download setting in Drupal.

• For clean URLs, go to Administer → Clean URLs.
• For private downloads, go to Administer → File system.

Step 2: Install the modules

Download and install all the above modules. For help on how to install the image libraries, see GD2 or ImageMagick. Note that you only need one of them, and they may be already installed.

The ImageAPI module comes with two sub-modules. Enable ImageAPI GD2 or ImageAPI ImageMagick, depending on which library you want to use. Also, enable ImageCache UI.

Step 3: Check image toolkit settings

Go to Administer → ImageAPI. You should see that your image library is enabled. If it isn’t, now would be a good time to look into troubleshooting.

Step 4: Create an image preset

ImageCache uses presets for image editing operations. You can think of a preset as a list of changes to perform on an image. Each preset has a name that you choose, like “100×100″ or “cropped_avatar”.

Later on, you’ll choose which preset to use for avatars. You can use different presets for default, comment view and the user profile page.

You can also use presets for many other things like image gallery thumbnails, but that is outside of this tutorial.

To create a preset:

• Go to Administer → ImageCache
• Click “Add new preset”
• Give your preset a descriptive name and click “Save Preset”
• Add operations to your preset by clicking the operation names.
• Click “Save Preset” again.

For avatars, the operation you probably want is “Scale and Crop”. You’ll specify a size in pixels, and the images are scaled to fill that size. Whatever doesn’t fit is cropped out.

Step 5: Enable the preset for profile pictures

• Go to Administer → User settings
• Make sure “Picture support” is enabled.
• Choose the presets to use with profile, comment and default.
• Click “Save configuration”.

One more thing: User pictures don’t appear by default in posts and comments. To enable them:

• Go to Administer → Themes → Configure
• Check “User pictures in posts” and/or “User pictures in comments”
• Click “Save configuration”.

Step the last: Troubleshooting

Unfortunately, it’s my experience that ImageCache does not usually work on the first try. The developers of the module have a pretty good troubleshooting section, and there are also sprawling forum discussions.

One key thing to check is if ImageCache can write into your /sites/all/files directory. The module saves every edited image it creates there.

The issue is that documentation typically comes in two varieties:

• A friendly tutorial covering a simplified application
• Low-level class and method reference

Between the two, there’s a massive gap. Reading a one-hour tutorial simply doesn’t get you to the level where a class reference is a useful resource. It’s like handing a beginning English learner nothing but an abridged dictionary.

Depending on the project, the gap can be gaping open or patched with a mish-mash of wiki pages, blog posts and forums. You simply don’t know where to go to learn the next step.

To give a more concrete example, SproutCore uses something called views for building interfaces. They are basically classes representing different types of UI elements. Having read the tutorial, I’m filled with questions.

• How many different kinds views are there? I suppose I’ll have to go trough the reference looking for classes ending in -View.
• What do they look like? There’s a demo page, but all the class names aren’t listed.
• What are the semantics? If there’s a list view and a scroll view, does this mean lists don’t scroll by themselves?
• How can they be nested? Can I put, say, a toolbar in a modal panel?
• What are the rules regarding sizing? The tutorial casually mentions that some attributes are derived implicitly.

You get the idea. I’m sure the the answers are somewhere, but finding them is a needle-and-haystack job through disorganized posts and mailing lists.

This isn’t to complain about SproutCore specifically, the same issue exists in many other projects. (Drupal, I’m looking at you.)

There seems to be a force driving documentation to two opposite poles. On one hand, developers feel a duty to document (but not necessarily explain) their code. On the other, well-meaning people write the very lowest level of documentation – over and over.

So, what can be done? The one thing I’d like to see is more articles on how to get specific key tasks done. Resist the urge to write another “getting started” tutorial that touches on a little of everything.

The author promises to write some useful documentation for an open-source project in the next week.

Step 0: Design for 980 pixels and do nothing

The width of the iPad screen when held in portrait orientation is 768 pixels. Since many pages (like the NYT, which Jobs has demoed) are wider than that, the iPad scales down pages.

The NYT website is automatically scaled down to fit on the iPad screen.

The default viewport width on the iPhone and presumably the iPad is 980 pixels. This means that pages are zoomed out until 980 pixels of horizontal content are visible. Text will wrap at 980 pixels, and elements wider than that will require horizontal scrolling.

If your site is designed to work at 980 pixels wide or slightly less, you probably don’t need to do anything. (Apart from not using Flash and the like, but we won’t go into that now.) The site will look all right, although a little bit small.

Step 1: Adjust your viewport settings

What if you don’t want automatic zooming? Perhaps your site is designed for a specific size, or maybe it flexes for different sizes. You’ll need to adjust the viewport settings with a meta tag.

The City of Helsinki site is narrower than 980 pixels, causing unnecessary margins. Adjusting the viewport fixes this.

The viewport meta tag was introduced by Apple for the iPhone, and it has since been picked up by Microsoft for Windows Mobile and Nokia for Maemo. The tag is ignored by regular desktop browsers.

<meta name="viewport" content="width=device-width" />

The above example essentially sets the zoom level to 100 percent. More specifically, it makes Safari scale content so that the number of visible pixels matches the screen width. This is useful for flexible layouts, but you’ll have to make sure your site flexes to both the iPhone and iPad (see step 2).

A little gotcha is that Safari always calculates device width based on the portrait orientation. If you rotate to landscape, the content is not reflowed, but scaled up to fit the wider screen.

<meta name="viewport" content="width=800" />

You can also set the width to a specific number to fit content designed for that size. For example, a site designed to be 800 pixels wide can be scaled to show that exact width.

Be careful with using a number, however. If you set the number to 320 for the iPhone, the iPad will most likely obey this and scale your content way up.

<meta name="viewport" content="width=device-width, user-scalable=no" />

Adding user-scalable=no disables the pinch-to-zoom feature. This is useful for preventing accidental zooms and makes web apps feel more app-like. Still, consider if there’s something your users might want to see at a larger size.

Step 2: Make it flex

If you use the device-width keyword, you have to deal with greatly differing screen sizes. CSS media queries to the rescue.

CSS media queries allow you to specify completely different stylesheets depending on how large the screen is. You can have one stylesheet for the iPhone and other mobile devices, one for the iPad, one for desktops and so on.

An old CSS trick is to also include a print stylesheet, eliminating the need for a separate printer-friendly page.

<link media="only screen and (max-device-width: 480px)" href="small.css" type= "text/css" rel="stylesheet">
<link media="only screen and (min-device-width: 481px) and (max-device-width: 1024px)" href="medium.css" type="text/css" rel="stylesheet">
<link media="screen and (min-device-width: 1025px)" href="large.css" type="text/css" rel="stylesheet">
<link media="print" href="print.css" type="text/css" rel="stylesheet">

The above example gives a page four stylesheets, which are automatically switched based on need. The only keyword prevents non-CSS3-compliant browsers from picking up the smaller styles, while starting with screen makes them use the large one.

If you don’t include any media keyword or use all, every device applies the stylesheet. This is useful for global styles you want for each device.

You don’t necessarily need to split your CSS into separate files. You can also use media queries inside a single stylesheet.

@media only screen and (min-device-width: 481px) and (max-device-width: 1024px) {
	body{
		//iPad body styles go here...
	}
	h1{
		//iPad heading styles go here...
	}
}
//Styles for all devices go here...

In the above manner, you can write exceptions for a single screen size inside an otherwise shared stylesheet.

For more information about Safari, see the Safari Dev Center.

Oops, sorry. I got the date wrong, this actually happened in 2000. The rest of it is true. Except that the revolution fell pretty flat.

“We will look back in five years and be very excited about what has transpired”

In the late 1990’s, a group at Microsoft led by colorful characters such as Bill Hill was working on improving electronic reading. They produced two major contributions: the ClearType font smoothing technology still in use today, and a book reading application called Microsoft Reader.

Reader automatically formatted electronic texts and presented them one screen-sized page at a time. The interface was beautifully minimal, with as little distraction for the reader as possible. (Kindle for PC designers, take note.)

Microsoft reader on Windows 98 and Pocket PC 2000.

The application was first released in April 2000 for Pocket PC, the forerunner of today’s Windows Mobile. A desktop version followed in August, and a Tablet PC version two years later.

One very powerful feature in Reader was its annotation capabilities. The user could highlight text, add their own notes and mark any number of bookmarks. There was also a convenient index containing all the annotations for a book.

The books themselves were in the proprietary .lit format, an offshoot of Microsoft’s Compressed HTML Help. Microsoft published a plugin for converting Word documents to .lit, and there were also third-party tools.

Microsoft’s first partner in their book effort was Barnes & Noble, who would sell titles in .lit format. There was a DRM scheme which required users to authenticate devices using Microsoft Passport (later renamed .NET Passport).

Going quietly into that good night

Unlike other products in the Lost Tech series, Microsoft Reader never really went away. It was simply… neglected. You can still download it from Microsoft today, although the company doesn’t seem to remember it exists.

Reader was closely tied into Microsoft’s Tablet PC vision, which steadfastly refused to generate sales. One clue about the intended device was that the desktop version had portrait-only windows. You had to either deal with a lot of lost space on the sides, or hold your laptop sideways and rotate the app.

The last update for Reader on the desktop was published in 2003. There was a version made for the failed Origami form factor in 2007. It quickly went the way of the dodo along with the Origami.

Perhaps the most damning point about the saga is that when ebooks turned hot once more, nobody at Microsoft remembered that they already have a platform. Instead of dusting off Reader, Steve Ballmer stood at the 2010 CES demonstrating Kindle software.

Oh well, at least we got ClearType. It only took until 2007 for it to become default on Windows.

A: Try this URL pattern: http://en.wikipedia.org/w/index.php?action=render&title=Helsinki

Today, I spent over an hour looking for this little URL parameter trick. What led me astray was that it’s not in the most obvious place you’d look – the MediaWiki API. Sure, you can get a parsed article out of the API, but it’ll be wrapped in unnecessary XML and character escaping. This gives you just the HTML you need. (There will still be infoboxes and edit links, but those are page content.)

At WWDC 2007, Steve Jobs infamously announced Apple’s “pretty sweet” solution for iPhone development: web apps. The response was developer derision and an eventual native SDK.

Nevertheless, Apple has been steadily working on web technology and improvements for Safari Mobile. Among the App Store hoopla, they’ve built a nice (if largely ignored) secondary app platform.

The rundown

Here are the key web app technologies present into the iPhone today:

Web clips

Web apps can put their icon on the home screen and launch just like regular apps. You get a full-screen view with no Safari UI, and even a custom splash image while loading.

Offline

Apps can be cached for completely offline functionality – all you need is a manifest. You also get HTML5 offline storage for all your database needs.

Location

Apps have access to Core Location (with user permission) for GPS and Wi-Fi triangulation. Just ask for coordinates through a JavaScript API.

3D effects

Apple’s extensions to CSS give you hardware-accelerated 3D transformations. It may not be enough for realistic games, but plenty for smooth animation, UI work and transitions.

Dashcode: Apple’s best kept public secret

Apple has published a JavaScript framework for making iPhone-like web apps, and a visual IDE to go with it. Dashcode used to be an editor for Dashboard widgets, but now it makes iPhone apps as well.

Making apps with Dashcode involves no HTML or CSS. Instead, you get an Interface Builder -like tool for visually constructing your UI out of ready-made parts.

The code side of Dashcode resembles iPhone development, though in JavaScript instead of Objective-C and Cocoa Touch. A list view is connected to a corresponding view controller, for example.

Dashcode includes several templates for common app needs such as hierarchic navigation and flip-around settings views. Apple has not been too busy documenting the system, though.

So, I hear there’s this tablet coming up…

The Tablet is supposedly based on the iPhone OS, and its version of Safari will no doubt contain all the same technology as on the phone side.

Obvious advantages for the Tablet are a bigger screen and more power. Web apps will launch faster, and they will have more real estate for compelling user interfaces.

What could Apple add to the Tablet? Some possibilities are more APIs for internal sensors (acceleration, compass), access to local data (photos, contacts…) and support for new multi-finger gestures.

Some of these additions are more likely than others. Web apps are not high on Apple’s list of priorities, so chances of integration with local data might be low. On the other hand, new gestures seem likely, especially if they bring benefits to all sites.

In any case, I predict that in the next year, there will be several high-profile web apps tailored for the Tablet. Web apps may not get the same kind of media attention as the App Store, but more and more developers will discover it.

And of course, there’s the matter of Google’s all-web-all-the-time Chrome OS. Having web app developers on their side might be a pretty smart move for Apple.

When it comes to the size of the tablet, there are three things I consider fixed:

1. The iPhone has a resolution of 320×480 with a density of 163 PPI (points per inch).
2. The resolution of 720p HD video is 1280×720.
3. The tablet is widely rumored to have a 10.1-inch screen.

The third point needs to be taken with a pinch of salt, of course, but I’m not going to argue against it.

I’m going to assume that the tablet plays 720p video at native resolution. For a resolution of 1280×720 at a 16:9 ratio, a 10.1-inch screen has a density of 128 PPI. This seems reasonable, but I don’t think it’s quite right.

I think that the tablet is going to have the same 3:2 picture ratio as the iPhone. This will make it easier to port current apps to the tablet. Apple is also said to invest heavily into digital reading, and 3:2 seems a more comfortable ratio than the very wide (or tall) 16:9.

Let’s make the screen a little bit taller. The ratio is now 3:2 but native 720p video fits inside, letterboxed. This gives us a resolution of 1280×854. It may seem a little oddball, but Apple used to make PowerBooks with the same resolution. Density for a 10.1-inch screen is now 152 PPI, which is closer to the iPhone.

Resolutions of the iPhone, 720p video and my tablet estimate.

This is naturally all guesswork. A large high-density screen might be too expensive, or Apple might opt for a more common resolution for the sake of mass production. Still, this is my best guesstimate.

What about apps, then? I’m taking it as given that Apple is making a custom UI for the tablet and not running standard OS X applications. They’ll also want to have apps quickly, so repurposing existing iPhone apps make sense.

The tablet has several times the screen real estate of the iPhone – seven times as much in my estimate. I doubt that Apple is going to implement windowing on the tablet, so apps have to be made to fill up the screen.

The simplest solution is to scale up existing apps – made much easier if the tablet has the same image ratio. It’s a stopgap method only, as blowing up the interface is going to provide a poor experience in most cases.

The buttons from Twitterrific appear gigantic at tablet sizes.

Apple has been working on density-independent interfaces for years. Existing apps might automatically benefit from this through bigger, native-resolution fonts and controls. Any bitmaps, however, will appear jagged or blurry after zooming.

A bigger problem is that enlarged versions of small-screen interfaces make little sense. Elements will be uncomfortably large, and any touch gestures will have to be scaled up as well. It’s a different thing to swipe across a phone screen or a tablet.

Developers are going to want to tailor their apps for the tablet, and for that they need a UI toolkit. The UI conventions of the iPhone and the Cocoa Touch library are made for a phone-sized device. When kept the same size but spread across a bigger screen, they appear ridiculously sparse.

A mockup of the current iPod app adapted to tablet size, clearly not viable.

What new UI elements could Apple provide? Some obvious choices are Cover Flow and a tablet-friendly version of the grid views in iTunes and iPhoto. A version of the column view from Finder could be used in place of hierarchic lists.

On a higher level, tablet apps don’t need to consist of tiny screenfuls at a time, like iPhone apps do. There will be less navigation and more context – more onscreen at the same time.

All of this is known to Apple, and they’ve no doubt been busy at work creating new interface conventions for the tablet. They have to present an easy way for developers to shift into the tablet without dragging in too much of the phone or the desktop. It’s a tall order.

I can’t wait until Wednesday.