Important Notes and Disclaimer

All code is provided as is and without any warranty of functionality or fitness for a given task.

The framework is based on caffe (http://bvlc.eecs.berkeley.edu). The caffe framework can run entirely on the CPU or use GPU acceleration. If available, it is highly recommended to use GPU acceleration. By using GPU acceleration the computation times are drastically reduced by a factor of 20-100 (i.e. computations take minutes instead of hours).

Please file bug reports to maid@informatik.uni-freiburg.de including information about your system and hardware.

Content

Installation Instructions

Prerequisites

You need a computer for runnning the backend (Cell Net) and a computer for running the frontend (ImageJ with our U-Net plugin). You can run the frontend on the same computer as the backend if desired.

Backend (Server) requirements:

Frontend (Client) requirements:

Backend (Server) Setup

Server Setup for GPU

We recommend that you use Ubuntu 16.04 on a computer with a GPU with at least 6GB (e.g. Titan). If you have to use the old Ubuntu 14.04, replace the string "16.04" everywhere with "14.04". If you don't have a gpu replace "gpu" with "cpu" and skip the installation of CUDA and cuDNN.

We describe the setup process for the fictional user "maid" who wants to install the caffe U-Net backend in the directory "/home/maid/cellnet" on the host with IP "192.168.0.10" and hostname "backendserver". He runs both backend and frontend on the same machine with ubuntu 16.04 and a GTX 980M GPU with 8GB. You can watch a video tutorial describing every step of the installation at https://www.youtube.com/watch?v=HJAUooQZ3V8.

Installation of CUDA 8.0 and cuDNN 5.0

Download the CUDA 8.0 binary installer from https://developer.nvidia.com/cuda-downloads to /home/maid/Downloads.

Run the installer:

cd /home/maid/Downloads
sudo sh cuda_8.0.44_linux.run

Follow the instructions; It is sufficient to only install the CUDA 8.0 Toolkit.

Download the cuDNN v5.0 for CUDA 8.0 library for Linux from https://developer.nvidia.com/cudnn to /home/maid/Downloads.

cd /home/maid/Downloads
tar -xf cudnn-8.0-linux-x64-v5.0-ga.tgz

Set the environment by adding the following line to your .bashrc:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-8.0/lib64:/home/maid/Downloads/cuda/lib64

Installation of the Cell Net package

Download caffe_unet_package_16.04.zip and caffe_models_2d.zip to /home/maid/cellnet.

cd /home/maid/cellnet
unzip caffe_unet_package_16.04_gpu.zip
unzip caffe_models_2d.zip

Edit your .bashrc file to set up the environment for the caffe U-Net software (You can combine the LD_LIBRARY_PATH statement below with the one above):

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/maid/cellnet/unet_package_16.04_gpu/libs
export PATH=$PATH:/home/maid/cellnet/unet_package_16.04_gpu/bin

Test, if it works: start a new shell and execute

caffe

A usage message should appear on the screen.

Test whether execution over ssh works:

ssh localhost caffe

A usage message should appear on the screen.

Frontend (Client) Setup

  1. (optional) Check, if you can connect to the server and execute a program there:
  2. Installation of the FiJi U-Net plugin.

The plugin needs to be installed on the client computers only. It has no included segmentation capabilities, instead it interfaces existing caffe installations on the local machine or a backend server that is accessible via secure shell (ssh) and secure file transfer (sftp).

Using the FiJi U-Net plugin with the pretrained 2D Cell Net

Walk-through example

  1. Open a gray valued image, e.g. /home/maid/cellnet/caffe_unet_package_16.04_gpu/2dcellnet_finetune/valdata_rbc/48hr-003-DIC.tif
  2. Use Image->Properties... to set the pixel extents
    Unit:micrometer
    Pixel width:0.45
    Pixel height:0.45
    Voxel depth:1

    (For different images exchange width and height values by the actual pixel-size of your image)

  3. Setup a new segmentation with Plugins->U-Net->U-Net Segmentation Manager->New Segmentation
    Model:2D Cell Net v0*
    Weight file:/home/maid/cellnet/caffe_models_2d/2d_cell_net_v0_snapshot_iter_150000.caffemodel.h5
    Process Folder:/home/maid/cellnet
    Use GPU:GPU0
    Memory (MB):8000
    Host:192.168.0.10
    Port:22
    Username:maid
    Password:********

    *see below how to select the Folder to chose Model from

  4. Click "OK"

Segmentation progress will be shown in a new row of the job table. After the caffe_unet binary on the backend server has finished the "Cancel" button on the right will change to "Show". Click it to show the segmentation result.

U-Net Segmentation parameters

Model:
Use the "Select folder" icon on the right of the "Model:"-line to select the local folder containing the Cell net model (2D or 3D) (e.g. /home/maid/cellnet/caffe_models_2d). Then select the model you want to use for segmentation in the combo box.
Weight file: (Path on the backend server)
The weight file contains the weights of the trained Cell Net. The weights file must be available via the given path on the backend (server) running caffe_unet.
Process folder: (Path on the backend server)
During the segmentation process intermediate files will be created and stored in the given folder on the backend server running caffe_unet. These files are: the model definition, the normalized image data, and the segmentation result. After closing Fiji, these temporary files will be removed.
Use GPU:
Select the GPU that is used for the segmentation. In CPU-only mode, select "None".
Tiling Layout:
Depending on the selected model, there are various options of defining the tiling layout. If available (which is the case for 2D Cell Net) we recommend to use the "Memory (MB)" option with which you can define the available amount of memory on the GPU. The options "Tile shape (px)" and "Grid (tiles)" are always available and let you define the maximum tile size or the tiling layout. The given values are upper bounds and will be adjusted to appropriate network input sizes. "#Tiles" uses the given amount of tiles and automatically defines the tile shape for optimal performance. "#Pixels/Tile" let's you set the number of input pixels of each tile. See also "caffe_unet --help" for further details.
Use remote host:
Check this to use a backend server for segmentation. If you want to use the local machine for segmentation, please uncheck.
Host:
The hostname of the backend server on which caffe_unet is installed.
Port:
The SSH port of the backend server (Default is 22).
Username:
Your username on the backend server.
Password:/RSA key:
Your SSH password on the backend server.
Keep original:
If checked, the original image is retained, otherwise it is replaced by the normalized image in processing resolution.
Output scores:
If checked, the output scores of the network are output instead of the binary segmentation masks. The output scores contain as many channels as there are classes. The segmentation can be obtained through the scores via a pixel-wise arg max operation over the channels. Scores are especially useful during training and fine-tuning to get an idea of the training progress.

Troubleshooting

If using GPU acceleration, make sure the graphics card is being recognized. Execution of

nvidia-smi

should show a table indicating the NVIDIA GPUs installed on the system.

When logging in to the server, the command

caffe_unet

should display a usage message by caffe. If not, make sure the $PATH and $LD_LIBRARY_PATH environment variables are set correctly. The same usage message must also appear when you run the command with a non interactive shell from a remote server:

ssh user@backendserver caffe_unet

If .bashrc contains an expression such as

case $- in
*i*);;
*) return;;
esac

or

[ -z "$PS1" ] && return

all instructions after that line will be ignored, thus the specification of the environment variables must be placed before this expression. Ensure that no outputs to standard output are generated in your .bashrc in non-interactive mode, otherwise file upload/download via sftp fails!

The selected model file is uploaded to the backend server when starting the segmentation. The weights must already reside on the backend server at the given location. The process folder is created on the backend server on demand given sufficient user rights.