Linking OMERO with elabFTW

Introduction

Upload Images to OMERO

OMERO offers multiple ways to upload images to the server. To keep it simple for all users, this manual will focus on the desktop client, OMERO.insight. However, advanced users can refer to the official documentation for alternative methods, such as using the command-line interface (CLI) or OMERO.dropbox. This manual was adapted from the official documentation available in link .

Import data using the Desktop Client

Setup

To upload images to OMERO, the first step is to download and install the OMERO.insight desktop application. Follow these instructions for installation:.

Download the OMERO.insight client corresponding to your operating system at: https://www.openmicroscopy.org/omero/downloads

Windows

• From version 5.5.0, OMERO.insight comes with two installers: .msi and .exe.

• Click onto the downloaded .exe or .msi file.

• This will run the installer. The .exe file installs by default in the userspace apps folder. This can be changed during the installation process.

• The .msi installer will deploy the application in the Program Files folder of Windows. OMERO.insight is then available to all the users of the target machine. OMERO.insight installed in the userspace is only available to the user who did the installation. A Desktop icon and a new OMERO.insight Start menu item will be created.

Mac

• From version 5.5.0, OMERO.insight can be installed using an Apple DMG (.dmg) file.

• Mac OS X: Click onto the downloaded .dmg installer to start the installation.

• This will mount the DMG it to your Mac. The DMG mounts in two places: on your Desktop and in the Finder sidebar under your hard drive.

• Clicking either one of these opens the DMG file. When you open a DMG file, you will usually see two things:

  • the app itself.

  • a link to your applications folder.

• Depending on your settings, the Applications folder icon might not appear. In such case, drop the app icon into the Applications folder.

Linux

• Unzip the downloaded .zip file.

• Click on the omero-insight file to start the application.

Step-by-step

1. Open OMERO.insight and in the login dialog, click on the wrench icon .

2. This will open a list of servers to which you can connect. By default, only “localhost” is listed. Click on the plus icon to add a new line to the list and type into "repo.ipht-jena.de" .

   

3. When done, click Apply.

4. Log in using the username and password provided.

5. In OMERO.insight, click on the Importer Icon in the toolbar.

6. Browse your local hierarchy in the left-hand pane of the importer, select single images or whole folders and add these to the Queue by clicking on the arrow icon.

7. In the Import Location window, select the target Project and Dataset (existing or create a new one) to import to.

8. Note: If no Dataset is selected or created, a new Dataset will be automatically created and named after the folder containing the images to be imported.

9. Optional: Go to the Options tab

   • Click on to bring the Tag selection dialog.

   • Select the tag(s) on the left-hand side or create a new one.

   • Click to move the tag(s) to the right-hand side.

   • Click Save.

10. Click on the Import button in the bottom-right corner of the Importer window. You should see two progress bars for every image imported, Upload and Processing.

11. Note: The import of the next image in the queue starts immediately after the Upload of the previous one is finished. The Processing phase of the import is done on the OMERO.server only, and can be finished while the next image(s) is/are being uploaded to the server.

12. Once imports are finished, go back to the OMERO.insight main window and click the Refresh button above the right-hand pane. This will display the imported images inside the Dataset and/or Project you specified previously in the Import Location window.

Tags

Tags are essential tools for organizing and annotating bioimaging data in OMERO. Tags can be applied at various levels (image, dataset, or project) and help researchers categorize and easily search for specific data. Key-value pairs, on the other hand, allow for metadata enrichment.

In traditional file systems, organizing data through file hierarchies can become cumbersome, especially when searching for specific combinations of metadata (e.g., treatment, staining targets, time points). OMERO provides flexibility by allowing dynamic rearrangement of data using tags. This enables researchers to search for images across multiple datasets and projects without reorganizing the underlying file structure.

How to Annotate with Tags

Tags can be manually added in OMERO.web or OMERO.insight, during data upload using the OMERO importer, or automatically via the auto-tag function in OMERO.web (if installed by the admin).

1. Manual Tagging: Select images, go to the "Tags" section in the right-hand panel, and either select an existing tag or create a new one.

How to manage experimental data with tags

OMERO provides only two levels of hierarchy: Projects and Datasets (Figure 4). However, in our experiments, we often work with more complex folder hierarchies. How can we adapt a folder hierarchy to OMERO's two-level system? We propose using a tagging system suggested by Schmidt C. et al., which outlines three possible scenarios for translating a folder structure into OMERO's organizational framework (Figure 5). In the first scenario, the first level of subfolders under a project is translated into datasets, while each subsequent subfolder is added as a tag. In the second scenario, a dataset is created for each experiment, and tags are applied to capture any additional information. In the final scenario, datasets are created for the lowest level of subfolders, and tags are added to reference the parent folders. We recommend using the third scenario, as it offers greater flexibility and is easier for users to understand.

Key-Value Pairs

Key-value pairs are introduced in OMERO as a flexible method for annotating metadata. A key represents an object or concept, and the value provides specific details for that key. These annotations are part of the metadata and can be applied to images, datasets, and projects under the "General" tab.

There are three ways to annotate metadata in OMERO:

1. Manual key-value annotation involves selecting an image, dataset, or project and entering the key-value pairs directly under the "General" tab. These pairs can then be used in searches by entering the key followed by a colon and the value.
2. Bulk annotation using CSV files can be done if the appropriate scripts are installed. A CSV table is created with the first row containing the keys, and the first column labeled as "Image." The CSV file is then uploaded as an attachment, and the key-value pairs are applied using the "Key-Value from CSV" script.

Creating experiments in elabFTW

In elabFTW, there are two main types of entities (objects): Experiments and Resources. Experiments are owned by individual users, while Resources belong to a team.

Although similar, they differ in a few key ways:

• Resources can be booked.
• Experiment templates can be created by users.
• Resource templates (Resource Categories) can only be created by Admins.

Despite these differences, both share many attributes, such as tags, category, status, and links. This manual focuses on the use of experiments in elabFTW, and the details were extracted from the official user manual available at the following link: https://doc.elabftw.net/user-guide.html.

Experiments

To create experiments in elabFTW, you must first log in at "https://elab.ipht-jena.de".Then select Experiments from the main menu. It will display a list of Experiments, this is “Show mode”.

Experiments are listed by default mixed with experiments from other users in your team.

You can create an experiment by clicking the «Create» button on the top right of the screen. You will then be presented with an «edition» page; the two other modes being ‘view’: display a single experiment, and ‘show’: display a list of experiments.

An experiment is composed of:

• A title

• The main text content

These are the two required elements. In fact, only the title is required as the main content can be empty.

Edit mode

Toolbar

In edit mode, the top part of the page displays a toolbar with several actions available, the main ones are:

1. Go back to the listing

2. Go into “View” mode

3. Duplicate the experiment: will copy all content except attachments, change the Status to the default one, and set the Date to today’s date

4. Timestamp experiment: create a signed, legally binding snapshot of the experiment and store it alongside the attached files in an immutable archive

5. Export menu: export the experiment in various formats (PDF, ELN, CSV, etc…)

6. Pin the experiment: make it appear on top of the listing at all times

7. Lock/Unlock experiment: prevent further edition. If you’re the one locking it, you’ll be able to unlock it, but if it’s locked by someone else, you won’t

8. Ellipsis menu:

   - See revisions: the revisions system keeps track of changes in the main text of the experiment

   - See changelog: the changelog system keeps track of all the other changes of the experiment

   - Archive/Unarchive: allow hiding the entry from the default listing

   - Delete: perform a soft-deletion of the experiment: mark the experiment as deleted but keep it in the backend database

Date (Started on)

The date is today’s date by default. You can edit it as you wish. The effective creation timestamp is stored in the backend database in another (read-only) attribute.

ID

This attribute is not editable and corresponds to the unique (scoped to the instance), immutable ID of the entry.

Title

The title of your experiment. A duplicated experiment will have a «I» character appended to the title upon creation.

Status

This useful feature lets you set the ‘status’ of an experiment. By default you can have:

• Running (selected upon creation)

• Need to be redone

• Success

• Fail

These status can be modified completely by the admin in the admin panel.

Tags

The tags allow you to easily group experiments together. You can think of it as folders, but more powerful because each experiment can have many tags, thus allowing you to cross-search efficiently! All experiments with the same tag will be accessible by clicking this tag or searching for it. To validate a tag, press Enter or click outside the input field. It is saved immediately. The number of tags is not limited. Click on a tag to remove it (in edit mode). Tags are common to a team. Autocompletion favors the reuse of existing tags.

Permissions

The “Visibility” and “Can write” part allow you to control who can access this entry. Click the Edit button to display a menu and add or remove permissions.

Experiment (body)

This is where you describe your experiment and write your results. It is a rich text editor where you can have formatting, tables, colors, images, links, etc…

Inserting an image

To insert an image in the main text, simple drag and drop it in the text zone.

Tables

If you add tables you might want to sort the data in the table dynamically. eLabFTW got you covered. Sort icons will be displayed in view mode when so called header cells (<th>) are defined and a table is set sortable. The table should have column names in the top row. You can select the top row with the mouse by clicking the left mouse button on the leftmost cell and while keeping the mouse button pressed move the mouse to the rightmost cell. Release the mouse button. The top row should be highlighted now. Next, from the rich text editor menu select «Table» → «Cell» → «Cell properties». In the dialog change «Cell type» from «Cell» (<td>) to «Header cell» (<th>). Finally, you can activate the sorting by clicking the «sortable table» icon () in the tool bar. The icon will also indicate whether a selected table is sortable. After you saved the changes you can go to view mode and dynamically sort the table. The changed order is not stored in eLabFTW. Merged cells in the top/header row (colspan) and in columns (rowspan) are not supported.

Using LaTeX

It is possible to express mathematical/chemical notations in eLabFTW, and they will be rendered properly in view mode but also in the pdf export.

For this eLabFTW uses Mathjax with ams extension.

Try this (make sure it is not pasted between <pre> tags!):

$$\sum_{i=0}^n i^2 = \frac{(n^2+n)(2n+1)}{6}$$

Use one $ for inline mode and $$ for block mode.

Steps

Steps are a way to list the things one need to do during the experiment. So you can write several steps, and once they are done, click the checkbox to declare them finished. This is quite useful for long experiments spanning over several days, where the “Next step” will be shown in Show mode (index list), so you can see at one glance what is the next thing to do for this particular experiment.

Note that you can also declare steps in a template.

Linked items

This field allows you to link an item from the database. Just begin to type the name of what you want to link and you will see an autocompletion list appear. Select the one you want and press Enter. The number of links is not limited.

This feature can also be used to link an experiment to a particular Project. If you have a «Project» Item Type and have a Project item in your database, you will then be able to see all experiments linked to this project by clicking the Link icon.

Linked experiments

Same as above, but for experiments.

Attach a file

You can click this region to open a file browser, or drag-and-drop a file inside. The file size limit depends on the server configuration, but there is no limit on file type. If you upload an image, a thumbnail will be created. There is no limit on the number of files you can attach to an experiment.

Some files are recognized by eLabFTW:

• molecules files such as cif, pdb, sdf, mol files, they will display the molecule in 2D or 3D

• DNA files such as fasta, gb, ape, dna, gff, they will display a fully featured viewer

• images such as png, jpg, gif or tiff will get a thumbnail

• pdf files also get a thumbnail and can optionally be included in pdf exports

Ellipsis menu (the three dots on the top right)

The Switch Editor entry will switch from the WYSIWYG editor (TinyMCE) to the markdown editor. And the Delete entry is to remove the experiment.

When you are done, click the «Save and go back» button.

You are now in view mode.

View mode of experiment

In the view mode, you will find an actions button bar in the upper left part:

1. Go back

Go back to the listing.

2. Edit

Switch to edit mode.

3. Duplicate

Duplicating an experiment allows you to quickly create a new entry with the same Title, tags, body and links, but with today’s date and a running status. Uploaded files are not duplicated. A «I» character will be added to the title to denote that it is a replicate.

4. Timestamp

When you click this button, a timestamp archive is created. Timestamping an entry involves generating a full JSON export of the entry and creating a cryptographic hash of that data. This hash is then sent to a trusted third party: the TimeStamping Authority (TSA).

The TSA acknowledges the existence of the data and sends back a signed token, which serves as proof that the data existed at that specific time. This process follows the RFC 3161 standard for Trusted Timestamping.

The timestamped data and its token are then saved in the “Attached Files” section of the experiment as a zip file. This file is initially in an “Archived” state, meaning it is hidden from view by default. To view archived files, click the “Show Archived” button on the right side of the “Uploaded Files” section in edit mode:

This timestamp archive is immutable and cannot be modified or deleted.

5. Bloxchain timestamp

This button, representing blocks, will do the same timestamping as above, except it will use the blockchain technology and the service provided by the BloxBerg consortium. You can learn more about it here: BloxBerg website.

6. Export button

The Export menu allows you to save the entry in different formats. The term “Long term storage” refers to the PDF or the PDF contained in the zip archive being of a particular kind: PDF/A, an ISO-standardized version of the PDF format. It is a PDF format designed for long term storage, but transparent PNG will appear with a black background, so they are no longer the default PDF format, but an option. The PDF/A will also include the changelog, unlike the normal PDF export. It is the PDF format used for timestamping.

The ELN format is a new file format based on RO-Crate specification, containing a special file (in JSON-LD) describing the contents of the dataset (one or several experiments). It is a format designed and promoted by The ELN Consortium, an association of several ELN vendors that agreed on an interchange format for export/import of datasets. You can learn more about it here: TheELNConsortium on GitHub.

7. Toggle pin

Clicking this icon will make this entry appear on top of the listing (pin entry).

8. Toggle lock

The lock icon allows you to lock the entry to prevent further editing. If you lock it yourself, you can later unlock it, but if it is locked by an admin, a user won’t be able to unlock it.

9. Ellipsis menu

Three dots speak volumes,
Unveiling hidden options,
Ellipsis unfolds.

elabid

In the bottom right part of the experiment, you can see something like: «Unique elabid: 20150526-e72646c3ecf59b4f72147a52707629150bca0f91». This number is unique to each experiment, and immutable (won’t ever change). You can use it to reference an experiment with an external database.

Comments

People can leave comments on experiments. They cannot edit your experiment, but they can leave a comment.

Linking data

Linking data throughout the research process is essential to uphold the FAIR principles of data management [2]: Findable, Accessible, Interoperable, and Reusable. Currently, there is a significant gap in the availability of a system or software capable of linking data from various bioimage sources. To address this, we propose LEO (Linking Electronic Lab Notebooks with OMERO), a web-based solution. While LEO was originally designed to integrate only electronic lab notebooks (ELNs) with the bioimage repository OMERO, it has now evolved into a cloud-based system that allows the integration of additional data providers through a plugin system.

LEO operates via two main workflows (Fig. 13). First, users can define and establish links between data elements from different providers through an intuitive interface, with these connections stored locally. For example, an experiment recorded in an ELN can be directly linked to one or multiple images, projects, or datasets in OMERO. Second, users can easily retrieve and visualize these established relationships, with a summary displaying essential metadata from each provider.

LEO can be acces by using the next link: "https://leo.k8s.photonicdata.science/"

Add metadata