0. Structure of this document

• The stucture is not final
• Under each object with bullets may be listed sub-objects, Description, Interaction and Graphics Design. For each of these there may be more than one bullet.
• Tags - tags are used for outlining a group that should leave feedback on specific question. This tag may be:
  • [shift] for people from ShiftGlobal
  • [dev] for developers
  • [devid] for a specific developer
• Priorities - a tag with a digit 0-9. 9 means highest priority.

1. Aspects of the Sophie UI

1.1. Graphic look & feel (general)

I think the graphic look is generally good & we like it. The gray windows and blue details look nice.

1.2. Interactions (general)

Basic questions that need to be worked out:

• [dev]Drag and drop interaction can be found here GlobalSpecification/DragNDrop. Currently it is being refactored. Groups with elements (objects) that behave the same will be introduced in order to be able to describe each element behaviour and to have consistancy.
• [shift][dev]Number of halos on frames: ideally, there would be a balance between easily accessible functionality & less potential for confusion.

1.3. Common elements

• Flaps
  • For organizing the functionalities, we will use flaps. Flaps are parts of the layout.
  • Currently considered are the left and the right flap. Bottom flap is proposed for timeline editing.
  • I think we're in basic agreement that things used to create books (resources, frames, etc) appear in the right flap, while things related to the structure of the book (page thumbnails, list of books) appear on the left side.
  • We should hammer down exactly what the tabs are and what they are for - Sophie 1 had tabs with the vague names "tools", "library", "resources" which proved to be confusing.
  • Flaps contain palettes.
  • [shift]Graphics Design
• Palettes
  • Palettes look like windows and are attached to tabs.
  • [milo] - Detaching palettes and/or moving them to other tabs.
  • Controls
    • List - a lot of palettes in Sophie contain list control. List contains list items. List items are selectable and draggable.
      • [shift]Grapics Design
        • List items - we should have several types of them - Text, text with icon, thumbnail with text
      • [shift]Interaction Design
        • List items - will all of them be draggable? For example, the plugin configuration palette, list items should not be dragged to the page work area. How should we show this to the user?
      • Examples of list palettes
        • Page Preview palette - Shows thumbnails of the pages. List items are draggable. Do we need text (page number) in the list? The idea about showing that the page has flow frames is great, but does not cover all the cases (the frame can be chained with another frame on previous page for example). There should be one row for each page thumbnail.
        • Configuration palette - Presents plugins as a list with buttons for configuration and about box. Can be with icons for each plugin. What size?
        • Embedded Books Palette - shows books, that are embedded in this book.
        • Opened Books Palette - shows currently opened books.
    • [shift]Buttons - How will we approach buttons? We can do them inside the list (for list palettes), but it won't allow multiple selection manipulation, or we should think of good way to do it. See attached images attachment:ListPalette.png​ attachment:ListPalette2.png​
    • Meta - meta palettes will be used for displaying text and other information. Meta palettes should contain editable and non-editable fields.
  • Graphics Design - we currently have a realization of the palettes, see attached image
• Menus
  • Menus hold functionalities that cannot be triggered elsewhere, functionalities that can be accessed only by mouse (drag and drop) or keyboard shortcuts.
  • Menu items can only be clicked.
  • [shift]Graphics Design - we can use the operating system menu styles, but it may be inconsistant with the look and feel of the application. Windows users may feel more comfortable (not sure about Mac users). We can also use icons and text for the menu items. Currently existing menu items are listed here
• Halos and HUDs - Halos are little icons that trigger different functionality. Currently considered are two types of halos
  • [dev]Clickable halos - most of the halos are clickable. The behavior that is currently accepted is that clickable halos invoke HUDs. What about delete frame halo?
  • [dev]Draggable halos - halos that should be clicked on and dragged to trigger their functionality. Currently considered are Move, Resize and Rotate halos. Their functionality is overlapped with the position halo and HUD. Is the idea of having 8 points for resize of frames good? What about the move halo? Can we say that draggable halos will be one color, green for example?

• [shift]Graphics Design
  • We need to show somehow hint the user which halos are draggable and which are clickable.
  • One thing that would be extremely useful would be easily understood icons for the halos. These were vague at best in Sophie 1; this could be improved tremendously. This is connected to the arrangement of what's in each HUD; we can't have clear icons for halos if the HUDs contain a large variety of functionality.

• Floating dialogues
  • Some functionality in Sophie 2 is going to be in floating dialogue boxes - when the user presses control/command-F, for example, a window will pop up asking what you want to search for, like in other applications. This isn't how this was done in Sophie 1 (that was in the palettes); it would be useful to see how this could look.
• Tooltips
  • Graphics and Interaction design - tooltips appear when mouse is over an element for fixed amount of time and disappear on mouse leave.

1.4. Specific functionalities

• [shift]Timelines
  • Interaction and Graphics Design - The way timelines were done in Sophie 1 wasn't wonderful and could probably be reworked. One issue: in Sophie 1, every frame on a page had a timeline halo that was dragged to the timeline to move the frame to the timeline. (The frame itself wasn't dragged because the author presumably had positioned the frame on the page first.) This got the job done but it was confusing: most of the other halos did not behave in this way.
• [shift]Embedded books
  • The way embedded books were handled in Sophie 1 - particularly the editing of embedded books - was confusing and needs to be rethought. (This has not been implemented in Sophie 2 yet.)
  • [milo]Graphics Design
  • Interaction Design - The main issue here is the problem of focus: you can have two books on the screen at once. In the Sophie 1 design, there was confusion because it wasn't clear what the status bar controls referred to (almost always the parent book, I think) and what the content of the palettes referred to (usually the selected book). [dev]We could probably darken the parent book when an embedded book is selected to show which one can be edited in the moment.
• [shift]Templates
  • The idea of page templates is confusing to a lot of people; a way to implement this that would be quickly understandable would be fantastic. In Sophie 1, users ended up having to spend a lot of time in the page structure palette figuring out which elements on the page were part of the page template
  • [dev]Explanation of templates in Sophie 2
• Reader Application
  • The author application contains reading functionality; the UI of this shouldn't be too different from the authoring UI. The reader application, however, needs to have its UI thought through more carefully: many of the people using Reader won't have used Sophie previously. Because it's their first introduction, it needs to be easy to use; it should also be inviting.

(If we're planning on having Reader work in a browser as an applet, does all reader functionality need to be confined to a single window?)

1.5. UI Structure

Layout tree

• VlEngine
  
  • AppMainWindow
    
    • Description Application main window contains all of the layout elements and book windows.
      
    • Graphic Design
      
    • Interaction Design Main window has standard OS controls for minimize, maximize and restore. Dialogs that are currently considered block the Application window disallowing interactions with it. Main window is drop target for Drag and drop (See InteractionAndGraphicsDesign/DragNDrop). One issue with Sophie 1 was that Mac Sophie used Mac menu bars and not the application menu bars; the Linux & Windows versions used the application menu bars. Sophie 1 was also limited to just the application window, which is not a limitation in Sophie 2.
      
  • LeftFlap
    
    • Description The left flap is generally concerned with the structure of the book. In the left flap the use can find a list of open books, thumbnails of all the pages in his current book, and a list of the timelines in the current book. This flap should contain non-editing palettes in order to be used in the Reader app too.
    • Graphics Design
    • Interactions Design
    • ConfigTab
      
      • Description This tab contains the palettes showing what plugins Sophie is using. It's worth noting that these palettes are not going to be important to most users when making books; maybe this functionality could be hidden in a floating window called from the menu bars rather than appearing in a palette?
        
    • Description
    • Graphics Design
    • PluginsPalette
      
      • Description This palette shows a list of all the plugins in the working copy of Sophie 2.
        
      • Graphic Design This is a list palette.
        
      • Interaction Design Selecting a plugin shows metadata about the plugin in the PluginAboutPalette.
        
    • PluginAboutPalette
      
      • Description This palette shows information about the plugin selected in the PluginsPalette .
        
      • Graphic Design This palette displays metadata; for now, it's a paragraph of text. This plugin needs to be near the PluginsPalette to be comprehensible; these two palettes could possibly be combined?
        
      • Interaction Design Each plugin should have an "enable" and "disable" button. Not all plugins can be enabled or disabled; some are necessary.
        
  • BooksTab
    • Description This tab contains palettes having to do with book management. Much of this functionality is duplicated by the book tab bar at the top of the application window.
      
    • Graphics Design
      
    • OpenBooksPalette - Shows currently opened books. The active book should be marked somehow
      
      • Description This palette lists all currently open books in Sophie. This palette does duplicate the functionality of book the Windows menu (which lists all books) and the book tab bar, which shows all open books.
        
      • Graphic Design This is a simple list palette; it would be useful if it showed metadata for open books (preview, number of pages, size). If this were changed from a list palette to a tree palette, it could show embedded books as well & integrate the functionality of the embedded books palette.
        
      • Interaction Design Clicking on one of the books should open the book for editing.
        
    • EmbeddedBooksPalette Shows list of books that are embedded in the currently active book
      
      • Description The embedded books palette in Sophie 1 listed all the embedded books in the currently selected book. Books were listed by name and with a preview icon.
        
      • Graphic Design This is a list palette: essentially a list of books. Metadata would be useful: in Sophie 1, books in this palette could be either linked (where the book exists somewhere else) or fully embedded (where the book is part of the parent book's file system). This wasn't made clear, although it has ramifications for the user.
        
      • Interaction Design In Sophie 1, the icons of embedded books could be dragged from this palette to the page of an open book to make the embedded book appear on a page; the icon of the embedded book could also be dragged from the palette to the third field of a link HUD if the action was to open an embedded book in a new window. Neither of these interactions was particularly good. We need a clear way for users to link to embedded books; we also need a good for one book to be embedded in another book. If books could be dragged from the books palette to the embedded books palette, this might solve that problem. It might make sense to treat embedded books as resources and move them into the resources palette, rather than keeping them over here.
        
  • PagesTab
    
    • Description - The pages tab contains palettes that have to do with pages in a Sophie books: the page preview palette and the page structure palette. The page structure palette is minimized by default to just its title bar.
    • PagePreviewPalette
      
      • Description The page preview palette shows thumbnails of all the pages in the book.
        
      • Graphic Design The page preview palette should show the page in thumbnail; the thumbnail should be resizable. Each page should also have a page number next to it.
        
      • [5][shift]Interaction Design Clicking on a page selects that page and goes to that page in the book. In Sophie 1, page templates were dragged from the page template palette to the pages to apply page templates; if pages were multiply selected, dragging a page template to one of the selected pages will apply the page template to all selected pages. Pages could be inclusively selected by shift-clicking, or multiply selected by control-clicking. In Sophie 1, new pages were added to the book by dragging a page template between existing pages or before the first page or after the last page. This could be done in a different way.
        
    • PageStructurePalette
      
      • Description The page structure palette displays a tree of every object on the page as well as the page template being used. Controls allow moving objects from the page to the page template and vice versa, as well as saving the current page as a page template. Checkboxes allow controlling the visibility of objects on the page. Selecting an object in the page structure palette's tree selects the object on the page and vice versa.
        
      • Graphic Design This is a variation of the list palette, the tree palette. This palette makes the hierarchy of objects comprehensible to the user; it displays the z-order of the objects and which are on the page template and which are not. There is a lot of information crammed into a small space here; this needs to elegantly represent what's on the page.
        
      • Interaction Design Selecting an object brings up options: whether to move it to the page template or not, whether to make it invisible. This needs to be done better than it was in Sophie 1. It would be nice if objects could be dragged in the tree to change z-order; representing grouping here would also be nice.
        
      • Description - The page structure palette shows the user a list of all the objects on his current page. When he selects something on the page it is selected in the page structure palette’s list, and vice versa.
      • Graphics Design ? Since we don't have clear concept on this one, here is presented the Sophie 1 look of the page structure palette
  • ToolsTab
    
    • Description This tab contained palettes in Sophie 1 that didn't fit anywhere else - search, replace and spellcheck functionalities. There are probably better ways that things could be organized.
      
  • SearchPalette
    
    • Description - The search & replace palette lets the user search for words or phrases in the text fields of his book and replace them. Enter a phrase to search for and one to replace it with; click Replace to replace the first result, or All to replace all occurrences. If there are results, they are listed below, and clicking on them will go to the page in the book where the text appears.
      
    • Graphic Design This is a modified list palette.
      
    • Interaction Design Clicking on the text results will go to the text results. This palette would work as a floating window called from the menu bars.
      
  • SpellcheckPalette
    
    • Description This palette is used for spellchecking; buttons allow you to find the next misspelled word & select replacement spellings, if they exist.
      
    • Graphic Design This palette contains several buttons and a list of spelling suggestions.
      
    • Interaction Design Clicking the Next button moves the book to the next page with a misspelled word. This palette could be a floating window called from the menu bars.
      
    • Description - The spellchecker palette allows to spellcheck the book from the current flow. To start it, click in a flow and press the Next button.
• TimelinesTab
  
  • Description The timelines tab/palette in Sophie 1 listed all the timelines in the book and had two buttons allowing the creation of single-page or multi-page timelines.
    
  • Graphic Design This is a list palette. This probably doesn't need to be a whole tab of its own; it could go in the styles tab where the page templates are, for example.
    
  • Interaction Design Clicking on a timeline will open it in the timeline flap. Timeline creation can probably be done from a halo.
    
• AnnotationsTab
  
  • Description The annotations tab mostly is used in the Reader application, but it also appears in the Author app.
    
• Graphics Design
• HighlightsPalette
  
  • Description Highlights are a list of places that have been highlighted in the book. Clicking on one of them will go to it.
    
  • Graphic Design This is a list palette; it's a list of the first five words in each highlight.
    
  • Interaction Design Clicking on a highlight takes you to that page. Sophie 1 never had a good way of highlighting; text was selected, then Edit > Highlight was selected in the menu. A better way to do this (a highlighter tool) would be helpful.
    
• StickiesPalette
  
  • Description This palette lists all stickies in the current book; the new sticky button makes a new sticky.
    
  • Graphic Design This is a list palette; it's a list of the first five words in each sticky.
    
  • Interaction Design Clicking on a sticky goes to that page. Clicking on the new sticky button makes a new sticky.
    
• AudioAnnotationsPalette
  
  • Description This palette lists all audio annotations in the current book; the new audio annotation button makes a new sticky.
    
  • Graphic Design This is a list palette; it should show each annotation by time and page number.
    
  • Interaction Design Clicking on an entry should go to that page. Clicking on the new audio annotation button should allow the reader to record a new audio annotation.
    

• AppDocumentsDesktop
  
  • BookDesktop - The Book Desktop is a kind of book that is inside the workspace. It can contain different resources that can be used in other opened books.
    
  • BookDocumentWindow
    
    • Description Book Document Window is responsible for displaying a single book. This exists in Author App only. At the bottom of the window is located the book panel.
    • Graphic Design
    • Interaction Design Book Window has standard window controls and behavior. Scrollbars appear when they are needed. Scrolling may be done via mouse middle click.
    • AuthorPageWorkArea
      
      • SimpleSceneVisual
        
      • FrameMoveHaloMenu
        
      • FrameMoveButton
        
    • ReaderPageWorkArea
      
      • SimpleSceneVisual
        
      • FrameMoveHaloMenu
        
      • FrameMoveButton
        
    • BookBottomPanel
      
      • Description Book bottom panel is currently considered to be at the bottom of the book window. It may contain different controls depending on installed plugins. Default controls are listed below.
      • Graphic Design
      • Interaction Design will this be detachable?
      • ZoomPercentagePanel - Dropdown menu that allows typing in. Bound control - checks immidiately and if the value is correct, applys the change without confirmation. Has default values (25%, 50%, 75%, 100%*, Fit, Fit to width, Fit to height.
        
      • NavigationPanel - Has previous and next page buttons and page number field (bound control).
        
      • QuickSearchPanel - Text field for search in the current document. Available in both Reader and Author modes. Behavior is undefined yet.
        
      • ReaderModeCheckBox - Probably checkbox idea will be dropped and a button will be introduced since the book is copied internally and displayed in a new window.
        
      • ShowFramesCheckBoxPanel - Shows frames bounds on the current book. Active only in author mode.
        
• AppMenuBar
  
  • FileMenu
    
    • NewBookItem - Toggles Book Properties dialog for creating a new book
      
    • OpenBookItem - Triggers File Dialog to choose a book to open
      
    • SaveBookItem - If the book exists, saves it. Otherwise acts like "SaveBookAsItem"
      
    • SaveBookAsItem -
      
    • SaveBookAsTemplateItem
      
    • ImportItem
      
    • ExportItem
      
    • BookPropertiesItem
      
    • CloseBookItem
      
    • ExitItem
      
  • EditMenu
    
    • UndoItem
      
    • RedoItem
      
    • CutItem
      
    • CopyItem
      
    • PasteItem
      
    • DeleteItem
      
    • SpellCheckItem
      
    • CopyLocationOfSelectionItem
      
    • ToggleRulers
      
    • GroupItems
      
    • BreakGroup
      
    • EditGroup
      
    • SetBookTitleItem
      
    • SetPageSizeItem
      
    • ShowConnectionsItem
      
  • InsertMenu
    
    • InsertImageItem
      
    • InsertTextItem
      
    • InsertMediaItem
      
  • WindowMenu
    
    • CascadeItem
      
    • TileItem
      
    • DocumentItem
      
  • HelpMenu
    
    • HelpContentsItem
      
    • AboutItem
      
    • BugReportItem
      
• RightFlap
  
  • LibraryTab
    
    • Description The library tab contains Sophie "primitives": different types of frames and book templates. "Library" is not a great name for this (also too similar to "resources"); something better should be imagined.
      
  • BookTemplatesPalette
    
    • Description This contains a list of all the book templates that Sophie can find. Dragging one out to the workspace creates a new book using that template.
      
    • Graphic Design List palette with preview of each book template.
      
    • Interaction Design Dragging book template to workspace creates new book.
      
• StylesTab
  
  • Description This tab contains page templates and paragraph styles.
    
  • Graphic Design This tab contains list palettes.
    
  • Interaction Design The page template palette was the only one of these palettes that was used much in Sophie 1 - templates were dragged from this palette to the page to apply page templates or to add new pages. This was opaque to the user; a better way to apply page templates should be implemented.
    
• ResourcesTab
  
  • AllResourcesPalette
    
    • Description This palette shows all of the resources in the current Sophie book, divided by type.
      
    • Graphic Design This palette is a modified list palette: the list is separated by tabs of media type. We need to find a way to get a large amount of information into a very small space, as users will have a lot of resources in some books.
      
    • Interaction Design The user should be able to drag resources from this palette onto the page of the current book. Ways for the user to organize resources will be needed.
      
  • ResourcesMetaPalette
    
    • Description This palette shows metadata about the selected resource in the AllResourcesPalette.
      
    • Graphic Design This palette should should a preview of the resource as well as the filename, size, and possibly other details.
      
    • Interaction Design This palette only displays information.
      

• BottomFlap
  
  • Description In Sophie 1, timelines were edited in the bottom flap, which only appeared when a timeline was open. This is maybe not the best way to do this; a timeline-editing mode might be better, or a free-floating timeline window. The timeline editing area needs to be as wide as possible; it might make sense to have it as a separate window.
    
  • Interaction Design In Sophie 1, the timeline window could only open one timeline at a time. This might be reconsidered?
    
• TimelineTab
  
  • TimelinePalette
    
    • Description The timeline palette is where timelines are edited; it contains channels and controls. More detailed discussion of timelines can be found here.
      
    • Graphic Design The timeline interface is different from any other part of the Sophie interface; timelines are a complicated concept to explain in the context of electronic book making. The timeline interface, then, should look as much like the rest of Sophie as possible. Most of the people using the timeline interface in Sophie will not be sophisticated users of a/v editing programs; it should look simple and unimposing.
      
    • Interaction Design The main area of concern in the timeline in Sophie 1 was the timeline slider, which proved difficult to use because it functioned both as a zoom and as a scroll bar. These two functionalities should be separated. Another problem is that of indicating when an event on the timeline is playing and when it is visible.
      

Book Elements

Page

• Frame
  
  • TextContent
    
    • ChainedTextContent
      
    • StaticTextContent
      
• Frame
  
  • ImageContent
    
• Frame
  
  • VideoContent
    
• Frame
  
  • AudioContent
    
• FrameGroup
  
• PageTemplate
  
• BookTemplate
  
• TextResource
  
• ImageResource
  
• VideoResource
  
• MediaResource
  

Halos

• FramePropertiesHalo
  
• FrameAppearanceHalo
  
• LinkHalo
  
• AnchorHalo
  
• CharacterHalo
  
• ParagraphHalo
  
• ChainingHalo
  

HUDs

• PageHud
  
• FrameHud
  
• AlignHud
  
• AnchorHud
  
• CharacterHud
  
• ParagraphHud
  
• LinkHud
  

2. Schedule

(This part of the doc lists when things should happens, milestones etc.)

2.1. Major Dates

2009-08-14: This is the first Beta release; most of the UI should be integrated into the product at this point so it can be critiqued.

2.2. Minor Dates

2009-05-13: When the first round of this document will be critiqued by Shift.

3. Comments

Media controls are not included in the layout tree.