1       General

It is to keep in mind that this user interface must work in 2 ways. One way is that the user directly interacts with the UI. The second is that the UI is controlled from an external program via a COM interface.

NOTE: All images in this document are not to scale, they are used to discuss the functionality behind them. Of course the final appearance should orientate on this specification.

 

2       Image buffer

Here all the image management will take place. The image buffer dialog will be realized as a dialog bar. I.e. it is either docked to the main window without caption or it is a non-modal dialog. It holds a tree control that represents the images and their affiliation to series. This image buffer can manage single images as well as image series. The following image will illustrate this:

The image buffer consists of 3 areas: In the upper area there is a tree control (B) that holds either the internal tree of images if tab image (A) is activated or some directory structure if tab Disk is activated.

All the images(files) of the selected tree item of (B) are listed in the section (C). This is a list of at least N entries. If there are less than N images (files) available, then the rest is added internally and marked as empty.

The list consists of 3 elements per line. To the left (D) there is an index that is unique and is build out of the position in the tree and in the list: The last number is the running index within the list. The number(s) before are created out of the tree as shown below:

1
            1-1
            1-2
                        1-2-1
                        1-2-2
2
            2-1
                        2-1-1
                                    2-1-1-1
                                    2-1-1-2
                                    2-1-1-3
etc.

In the list each entry has an icon (E) that specifies the type of data. There will be several icons necessary, like “empty”, “grey image”, “rgb image”, “spectrum”, etc.

The last item of each line entry in (C) is the name of either the image of the file(F). If the image is created new, then there is a default name created out of the directory in which the image is created and the position within the tree ( using 3 digit numbers with leading 0s ). It is possible that the names are changed later.

This tree control shall work as usual under Windows, i.e. clicks on ‘+’ or ‘–‘ will open or close the sub tree, respectively. Also the standard cursor operations will apply.

The image buffer will have a flap out button(G). This button opens or closes the info flap out.

2.1      Info flap out

The info flap out (H) is an expansion of the Image buffer dialog. This expansion holds a tab-control to select between the different information types ( e.g. user data, image data, … ).

This information is combined with the currently selected image. If a series is selected in the tree control ( e.g. TMV in the example above ) series specific data has to be displayed.

In the future a thumbnail image can be displayed here if a certain tab is selected and

2.2      Actions in the image buffer

2.2.1    Actions in the tree control

2.2.1.1  Drag and Drop

The user has the possibility to drag image files from the explorer into the tree control (B).

The user has to drop those images on a directory of the tree control. All the images are then placed in this directory starting with the first free element in the list. If the list is too small it will be expanded. The image names are created out of the filenames of the images. It must be ensured that the names are unique in each directory.

If in the example above 3 files are dropped on the folder TMV in the tree control, the images are placed at position 1-4, 1-7 and 1-8.

2.2.1.2  Left click on a directory

This activates the selected directory. In the info flap out ( if open ) information about this folder is displayed. In the image list (C) the images of this folder are displayed.

2.2.1.3  Right click on a directory

This opens a popup menu with some context sensitive actions like ‘rename/save all/properties’ etc.

2.2.2    Actions in the image list

2.2.2.1  Drag and Drop

The user has the possibility to drag image files from the explorer into the image list (C).

If it is a single image then the image is placed at the selected position, overwriting ( ask before ) a possibly existing image.

If multiple images are selected, then the first dropped image is placed at the selected position, the next image at the next position, etc. Always with an overwrite check. If 3 images are dropped on the list item 1-3, the images are located then at the positions 1-3, 1-4 and 1-5. The image names are created out of the filenames of the images. It must be ensured that the names are unique in each directory.

2.2.2.2  Left click on an image

This is a very sophisticated function.

If the button down and button up is on the image name ( is some dragging in between allowed ? ), then it is treated like a click on the image name and the image is activated in the list, the image info is displayed in the info flap out (if open ) and the image is displayed in the current viewport.

If the image is dragged, then the image will **NOT** be activated.

The image can be dragged into a viewport or some place where image actions will be performed. ( Where are the places e.g. for image arithmetics, saving, etc, where I can drop the image. To be designed!)

2.2.2.3  Right click on an image

This opens a popup menu with some context sensitive actions like ‘rename/save as/properties’ etc.

2.2.2.4  Double left click on an image

This will open a new viewport and display the image in it.

3       Image/Series save dialog

If a single image is to be saved, the standard file selector will be used. If a directory is to be saved, then a dialog that selects a folder on HDD has to open, where the user can specify the

4       Image/Series load dialog

Open the standard file selector with the possibility of multiple file selection and place the images in the currently selected folder. This corresponds to drag and drop of several files on a directory in the image tree.

5       Histogram monitor

The histogram monitor shall be of the same type as the image buffer dialog ( i.e. dialog bar element ).

The dialog is divided into 2 areas. The upper data display area (A) displays the histogram data as it was requested by some viewport action, whereas the lower control area consists of several buttons, check boxes, etc. to both manipulate the histogram data as well as the associated viewport.

The following actions should be possible:

Moving of the left and right vertical bar in the histogram data and display of the associated values below ( 163 and 3871 in the image above ).

Scale the image in the associated viewport according to those vertical lines.

Scaling is an image property, i.e. the adapted scaling must be somehow be stored with each image and every time the image is displayed again in whatever viewport, the same scaling must be adapted. Additionally this scaling info must be visible in the histogram monitor.

The area (B) will contain a tab control with the tabs “Mean/Deviation”, “Over/underflow” and “”Fixed values” and maybe more. Each of those tabs will show some controls to manipulate the scaling of the image. It must also hold a button to apply the changes to both the histogram markers and the image scaling.

Also a “default” button would be helpful.

Is here a good place to have some controls for special LUT creation, logarithmic scaling, etc. ?

6       Viewports

There are several types of viewports, depending on their usage.

A viewport is a window that has some image area on the left side and according to its purpose some or more controls on the right side.

Image viewports will be used to display images, acquire images and image series and perform a simple analysis of image data ( e.g. display pixel intensity at mouse position).

Power spectra viewports are used to display power spectra and to analyse them ( linescan, rotational linescan, Fourier filter ). Really ?

There will be some other viewports necessary, like viewports to display XCF data, to display the path of correlations, to display other data. This has still to be designed.

6.1      Image viewports

Each viewport holds a specific set of camera and TEM settings. If there is an action started from the viewport that involves either the camera(s) or the TEM (or both ) those settings are sent to the devices before those actions are performed. A simple change between viewports does not affect any of the devices immediately.

6.1.1    Layout

The image viewport window consists of 3 areas:

The top button in the control panel area (C) determines whether the flap out is in (invisible) or out (visible).

6.1.2    Images

Images are displayed in the display area (A). The scaling of the images is performed in the histogram monitor.

The scaling is an image property.

6.1.3    Control panel

The control panel holds necessary controls to work with the image viewport:

There are 2 camera settings available for quick access. There is a toggle possibility between them ( in the first run e.g. via radio buttons ).

The button “Exposure” will start a single exposure with the exposure time below. Is it possible to acquire dark images ? How ?

The button “Continuous” will start continuous exposures. When this continuous acquisition is active, the button has to indicate that: Its text will change to “Stop” and additionally the button background color will change to yellow. A click on the “Stop” button will terminate the continuous acquisition.

The edit field below the 2 buttons will receive the exposure time for the acquisitions. There have to be also some spin controls to manipulate the exposure time using the mouse, both in a continuous and in a smart manner ( like it is realized in EMMENU 3.0 ):

It is an important point to note that the exposure time is a viewport property, i.e. if the user toggles between the 2 camera formats, the exposure time must be stored/restored. If the user selects some camera format in the flap out ( see there ) a default exposure time is set.

There are 2 buttons ( exactly one is active ) that specifies the mouse actions:

The 2 modes are Pan and Zoom. In pan mode the mouse pointer can click into the image, grab it and move it around. In zoom mode a left click zooms into the image, making ( if possible ) the clicked coordinate the centre of the image, and a right click zooms out of the image.
The mouse pointer should also change over the image into either an arrow or a magnifying lens to indicate the current mode.

The checkbox “Power” will indicate that a power spectra viewport is opened ( if not already open ) that is logically attached to this viewport meaning that if new image data is displayed in the image viewport, the corresponding FFT is also displayed.

The lowest control in the control panel is informative: If the mouse in either mode ( buttons up ) is moved over the image(E), the image coordinate and the real pixel value are displayed here.

6.1.4    Camera/Tem flap out

This flap out is opened/closed with the button (B). It holds 2 tabs ( maybe more in the future ) that are labelled “Camera” and “TEM”. Here the viewport depending camera and TEM settings can be set/changed.

Tab Camera:
There will be 2 drop down controls that enable the user to select 2 camera formats. It is also possible to select the same format twice ( and e.g. use different exposure times later ). Those camera formats have to be defined in the global preferences settings.

Tab TEM: Still to be defined in design and behaviour.

6.1.5    Actions in the image viewport

6.1.5.1  Activation of the viewport

If the viewport is activated, the image buffer must display the directory and the image in the image list where both the directory and the image must be activated.

6.1.5.2  Resizing

The viewport can be resized. It must be ensured that the viewport has at least a minimal width that holds the control panel and ( if open ) the flap out and also a “small“ part of the image together with the scroll bars.
The minimal height of the viewport should be that at least a part of the image can be displayed together with the scroll bars.
In a minimal size viewport some or more controls might be inaccessible.

6.1.5.3  Drag and drop

It is ***NOT*** possible to drop a file from the explorer into a viewport, however it is possible to drag an image from the image buffer into the viewports display area, where the image will be displayed and the logical connection between the image data and the viewport is made.

6.1.5.4  Mouse move over image

If the mouse is moved over the image(E), the image coordinate and the real pixel value are displayed.

6.1.5.5  Left click in image

If the pan mode is active and the (possibly zoomed) image is larger than the display area (A), then the image shall be moved along with the mouse as long as the left button is down.
If the zoom mode is active then the zoom factor is increased and the newly zoomed image is displayed. The pixel that was clicked should then appear in the centre ( if possible ).

6.1.5.6  Right click in image

If the pan mode is active nothing will happen.
If the zoom mode is active then the zoom factor is decreased and the newly zoomed image is displayed. The pixel that was clicked should then appear in the centre ( if possible ).

6.1.5.7  Click in scrollbar

If the image area is larger than the display area the image is scrolled accordingly to the scrollbar actions.

6.1.5.8  Exposure

This button starts a single exposure with the specified exposure time below and the camera settings of the camera/tem flap out. The acquired image is displayed in the display area, scaled using the current histogram monitor settings.

The histogram and power spectra are also updated ( if option is selected ).

The acquired image appears also in the image buffer dialog. It either overwrites the current image in the buffer or it creates a new image. This behaviour has to be set in some option dialog. Is this a global or a viewport option ?

6.1.5.9  Continuous

This button starts a continuous series of exposures. The button will receive a yellow background and change its text into “Stop”. The images will be taken one after each other until the Stop button is pressed. Then the “Stop” button will receive its original background color and the text “Continuous”.

The acquired images appear also in the image buffer dialog. They either overwrite the current image in the buffer or create a new images. This behaviour has to be set in some option dialog. Is this a global or a viewport option ?

The histogram and power spectra are also updated ( if option is selected ).

6.2      Power spectra viewports

6.2.1    Layout

The power spectra viewports consist of 2 areas: The display area (A) with its scrollbars (B) and a control panel (C) with some power spectra controls.

6.2.2    Actions in the viewports

6.2.2.1  Activation

What shall happen, if the power viewport is activated ? Shall the corresponding image viewport be brought to the top and/or shall the image buffer display(=select) the image ?

6.2.2.2  Resizing

The viewport can be resized. The minimal width and height must be so that at least a small part of the image and the scroll bars must be visible. In a minimal size viewport some or more controls might be inaccessible.

6.2.2.3  Moving

Moving with the mouse over the power spectra image it will display informational data in the control panel like: x and y position, intensity, angle and distance to the centre ( if possible in calibrated data, otherwise in pixels ).

6.2.2.4  Left click in the image

If the image is larger than the display area ( either zoomed or not ) then the mouse pointer should change into a hand and move the image according to the mouse movement ( panning ). If the image is displayed completely, nothing will happen.

6.2.2.5  Right click in the image

If the click was into image data, then somewhere in the control panel area of the viewport there should be the output of the image’s x and y coordinates below the mouse as well as the corresponding pixel value ( real 16float data, not displayed scaled 8bit data )

6.2.2.6  Click in scroll bar

If the image area is larger than the display area the image is scrolled accordingly to the scrollbar actions.

6.2.2.7  Zoom -/+

This decreases/increases the zoom factor of the displayed data. The scrollbars are adapted according to the new zoom factors.

6.2.2.8  Scale L -/+

This magic number influences linear the scaling of the left part of the displayed spectrum. If it is changed, the image is updated immediately.

6.2.2.9  Scale R -/+

This magic number influences linear the scaling of the right part of the displayed spectrum. If it is changed, the image is updated immediately.

6.2.2.10               Scaling

There will also be some additional scaling possibilities available ( e.g. non-linear scaling, several predefined LUTs etc. ). This has to be defined.

7       Tools

There will be one toolbar ( several ? ) available that holds buttons to start a lot of tools: Auto focus, Linescan, Fourier filter, Tiling, Tomography, Image arithmetic and filters, Recorder, Calibration, etc.

Their behaviour and appearance should also be defined properly!

8       Toolbox

The tool box is a non modal dialog that offers buttons and other controls that correspond to the current action. It has to be defined in detail
a) when does the appearance of the toolbox change and what the toolbox really offers.

9       Overview window

The overview window is ( like in PSP ) an non modal dialog that displays the full image of the current viewport. If the image in the viewport is not displayed completely, a rectangle in the overview window marks the visible area. This area can be moved. According to this movement, the contents of the image area of the active viewport must be moved. Additionally there are zoon in and zoom out buttons in the overview window. A click on those buttons will change the size of the rectangle in the overview image as well as a zoomed display of the image data.

10  Drop areas

There must be available a lot of areas where the dragged images ( from the image buffer ) can be dropped, e.g. for saving the image, for the arithmetic operations, etc. This has still to be defined.

11  Menu bar

12  Viewport manager

There must be a viewport manager available. At least as an internal structure. The following actions must be possible with the viewports:

Those actions can be started from the menu, from a specific dialog or from a toolbar.

13  Preferences

13.1 Available camera formats

There must be a list of available camera formats that can be selected in the flap out of the image viewport. Such a list entry must consist of the following items:

·       A unique meaningful name

This list must at least contain one element with a dummy camera. The list can be expanded/reduced dynamically.

13.2 TEM control

In the preferences there must be a selection what parameters can be changed/set in the image viewports ( before an exposure is started ). Only parameters that are selected here will be visible in the TEM tab of the camera/tem flap out. There the user will specify what parameters are set to what values.

13.3 Viewports

There must be some settings for the viewport manager, like max. number of viewports, viewport type specific parameters, predefined viewports ( search, scanning, … ) etc.

13.4 Warnings

Enable/disable thousands of different warnings ( overwrite image in buffer, overwrite file, image is not saved, etc. )

14   IEMGUIStd and internal structures

The user interface will have a COM interface named IEMGUIStd. The user interface will be controlled either directly by user input or by calls of the methods and properties of this interface or both.

It must be ensured that there is no conflict and that the resources are protected against multiple access from different clients.

The interface, its methods and properties as well as the internal structures needed are specified in a different document.

15  Problems / Ideas

15.1 Image buffer

If the image buffer is realized as dialog bar ( i.e. docking ) then it has to be checked how the flap out technique is realized. Is it either (A) a new dialog /dialog bar element that is placed below the current dialog or (B) is the image buffer dialog resized.

(A) new dialogs means that this dialog will be active

(B) Resizing means that the whole docking area is resized, i.e. other docked windows will be rearranged by the mainframe.

15.2 Overlay

Shell we keep an additional overlay image for each image? If so, in what format ( pixel oriented, vector oriented ) ?

15.3 File format

We will use TIFF 16 bit or TIFF 8 bit to store images. Therefore we will also use our registered tags to store user data and other things like e.g. thumbnail images, etc.