Here are Dan's notes on some issues that came up in the Sophie 1 UI (from the end user point of view). Not everything here will happen with the Sophie 2 model, but some of these basic issues will still be around.

Dan.1. How things worked in Sophie 1

In Sophie 1, when a frame was dragged to a timeline it became an event: a rounded rectangle in a channel on a timeline. The edges of the event could be dragged back and forth to change how long the event stayed visible. In this diagram, Frame 1 would be visible from 1:00 to 2:00:

One effect of how this worked was that once a frame had been dragged to a timeline, it was hidden on the page: the frame would only be visible when the playhead of the timeline was. When the playhead in the diagram above reached 1:00, Frame 1 would be shown; when it passed 2:00, Frame 1 would be hidden.

This behavior isn’t very important for audio on a timeline, because usually the visibility of the audio file’s image on the page isn’t very important. It’s worth focusing on in two particular cases of timelines & visibility: movies and interactions with links. (A third case, toggling visiblity via the page structure palette, also existed; however, I’m going to ignore that for now, assuming that the page structure palette will work differently in Sophie 2.)

Dan.1.1. Movies on timelines

When a movie is dragged to a page in Sophie 1, it doesn’t start playing automatically: usually you see the first frame, a black rectangle which looks like other frames. This frame can be dragged around the page; in a finished book, this black rectangle may indicate that there’s a movie that can be played, especially if it has controls underneath it. When a movie is dragged to a timeline, the movie behaves in the same way that a regular frame does: it becomes invisible when the movie is not playing.

This wasn’t always what users wanted; often, they wanted a black frame to appear until the timeline made the movie actually play. In later versions of Sophie 1, control-clicking a movie event on a timeline would toggle its visibility: if an event had been control-clicked, it would stay visible the entire time it was on a timeline, with the movie playing when the playhead was over it. This was not the best interface design; a better way to do this would have been to make this a check-box in the time-based media HUD. But this design element further complicated the problem of timeline visibility: the user couldn’t tell from looking at the timeline, what would be visible when.

Another problem with the way movies were handled on timelines was that the interface confused visibility with duration. The author would often want to clip a movie (i.e., if the have a 10 minute movie, they want to show from 2:00 to 5:00); the timeline seems like the logical place to do this. When you dragged your 10 minute movie to the timeline, dragging the edges of the clip would allow clipping from the back (making it play from 0–8:00 rather than from 0–10:00) but not clipping from the front (making it play from 2:00–10:00 rather than from 0–10:00). The problem was trying to include too much functionality with too few controls. Eventually, the ability to clip movies and audio that were on timelines was added to the time-based media HUD on the frame on the page; this probably wasn’t the most obvious place to put it.

Dan.1.2. Links and show/hide actions

Visibility on timelines was also complicated if there were links with show/toggle/hide actions. These links would do the following:

• show: a frame is set to be visible regardless of previous state.
• hide: a frame is set to be invisible regardless of previous state.
• toggle: a frame is set to be visible if it is invisible or invisible if it is visible.

Show and toggle worked the same way as dragging an event to a timeline: making a frame the target of one of these links would make it invisible by default. This is where the conflict with timelines arises: if a frame that is shown or hidden by a link appears on a timeline, the timeline’s display of the frame’s visibility will not necessarily be correct.

Dan.1.3. Show/hide via page structure palette

Because of these conflicts, a checkbox for frame visibility was added next to each frame’s entry in the page structure palette. When a frame was added to a timeline, it was unchecked because the frame became invisible; when a frame was made the target of a show/toggle link, the visibility checkbox was also unchecked. In addition, the user could manually uncheck frame visibility.

In practice, this didn't end up working very well. Users tended to make frames invisible so that they could work on other frames on the page (the way that layers are hid in Photoshop or Illustrator); they were confused when the visibility changes were persistent. Probably visibility for frames and visibility for authoring should have been separated.

Dan.2. Ideas for how things could work

Channels in Sophie 1 weren't specific: an event could be in any channel that the author wanted, and could be moved from channel to channel. It might make sense in Sophie 2 to have a channel for each event on a timeline. This would allow more specific controls for visibility: a channel for video could have a control specifying whether the frame was always visible or not.

Dan.2.1. Changing the representation of the event on the channel

It also might make sense to get rid of the idea of using a draggable rectangle as the signifier of the event's visibility - maybe it should be limited to representing clipping of audio and video. GarageBand on the Mac creates a sub-channel if you want to adjust the volume of a clip; there's a draggable line that can be moved up and down to adjust the volume:

Something like this could be done to adjust visibility rather than to adjust volume - if it could be implemented like this, we could even imagine using this to create fadeins and fadeouts. When visibility is separated from the movie's representation, the movie could be clipped separately from adjusting its visibility.

Dan.2.2. Noting interactions with links

It might make sense to have the timeline check for interactions with links on the page. If a frame on the page is the target of a link that changes its visibility and that frame is also on the timeline, it might make sense to put a small warning (maybe on the channel), indicating that there might be a possible conflict. We can't fix all of the user's problems - it's the author's responsibility to make a book or page that works for the reader.