CellOrganizer for Docker

About CellOrganizer for Docker

CellOrganizer for Docker is an image with compiled binaries from the CellOrganizer functions

• img2slml, the top-level function to train generative models of cells, and

• slml2img, the top-level function to generate simulated instances from a trained generative model.

• slml2info, the top-level function to generate a report from information extracted from a single generative model.

• slml2report, the top-level function to generate a report from comparing generative models.

• slml2slml, the top-level function to combine models into a single model file.

The image also comes with

• Conda

• Jupyter

• and the Python packages

  • cellorganizer-python 2.9.0

  • ipywidgets 7.4

  • pandas 0.24

  • numexpr 2.6

  • matplotlib 3.0

  • scipy 1.2

  • seaborn 0.9

  • scikit-learn 0.20

  • scikit-image 0.14

  • sympy 1.3

  • cython 0.29

  • patsy 0.5

  • statsmodels 0.9

  • cloudpickle 0.8

  • dill 0.2

  • dask 1.1

  • numba 0.42

  • bokeh 1.0

  • sqlalchemy 1.3

  • hdf5 1.10

  • h5py 2.9

  • vincent 0.4

  • beautifulsoup4 4.7

  • protobuf 3.7

  • xlrd

About Docker

Docker performs operating-system-level virtualization. Docker lets us create and deploy a preconfigured image with the CellOrganizer binaries. This image can be used to spin containers that are ready to use and start testing.

To learn about Docker and how to use it, click here

About Jupyter

The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text.

To learn more about Jupyter, click here

Installing CellOrganizer for Docker

Setup

The instructions below describe

• How to install Docker, the virtualization engine that will run the container

• How to download the latest cellorganizer-docker image from Docker Hub, i.e. the docker images repository

• How to start a container from the Docker image

• How to connect to the container

• How to run some of the demos included in the container

Installing Docker

Before downloading the image and spinning a container, you need to install Docker. Installing Docker is beyond the scope of this document.

• To install Docker-for-Mac, click here.

• To install Docker-for-Windows, click here.

Getting started

Its a two simple step process, where we download the image from docker hub and then run a container with that image.

Run the following command to pull the image from docker hub.

• docker pull murphylab/cellorganizer-jupyter:latest

To download the run script click here.

• run the downloaded script (run.sh). This will start a container and the terminal will show the Juypter url.

Demos

There are several demos included within the CellOrganizer software bundle. These demos are intended to illustrate CellOrganizer’s functionality, and should be used to familiarize the user with the input/output format of various top-level functions such as img2slml and slml2img. Certains demos have been deprecated and will be removed in future versions of CellOrganizer.

Demo	Training	Synthesis	Other	Deprecated
demo2D00		True
demo2D01	True
demo2D02		True
demo2D03	True			v2.8.1
demo2D04	True
demo2D05	True
demo2D06		True
demo2D07		True
demo2D08	True
demo2D09	True
demo3D00		True
demo3D01		True
demo3D04		True
demo3D05		True
demo3D06		True		v2.8.1
demo3D07		True
demo3D08		True
demo3D09		True
demo3D10		True
demo3D12	True
demo3D19	True		Report
demo3D25		True
demo3D47			Model
demo3D48	True
demo3D50	True
demo3D51	True		Plot
demo3D52	True
demo3D53		True
demo3D55		True	Plot
demo3D57		True	Plot
demo3D58		True
demo3D59		True
demo3D60		True

Running CellOrganizer for Docker through Jupyter-Notebook

Once you have the Jupyter running, now you can start playing around with the demos. We have a set of sample demos that can help you get started.

Your Jupyter folder structure looks like this.

The run script had created a folder named mmbios2021 on your desktop. Use this shared folder to transfer files (if needed) between your system and the docker container.

• Use the Download_files notebook to get the latest images, models and notebooks.

• These are fetched from the web and kept under mmbios2021 folder on your desktop.

The demos use various image sets and models located inside mmbios2021/images and mmbios2021/models folder.

The demo notebooks are located under mmbios2021/notebooks and are further divided as:

• demo_notebooks

  2D A collection of 2D jupyter notebooks demos

  3D A collection of 3D jupyter notebooks demos

• workshop_demos

  A collection of jupyter notebooks used during mmbios2021 workshop

An example of a demo that trains a generative model from a series of .tif image files is demo2D01. To run a demo, simply click the run button at the top of the notebook.

This demo will save a folder param containing .mat files as well as a .mat file lamp2.mat to the same directory (/home/cellorganizer/demos/2D/demo2D01). These .mat files contain information characterizing the trained generative model.

An example of a demo that produces simulated images from a trained generative model is demo2D02.

This demo will save a folder img containing these simulated images to the same directory.

To export generated data out of the container, click the files in the directory that will be exported and click download.

Important Docker Configurations

Some demos are compute heavy and might need more compute resource than which is allowed by default via docker. To increase RAM memory allocation and number of CPUs. Do the following steps :

• alter these two flags –memory=”8g” –cpus=2 to change the memory and cpu allocations.

Mac

Docker > Preference > Resources and change the default allocations.

Ubuntu

Check the docker documentation to see, how you can use commands to set various parameters. https://docs.docker.com/config/containers/resource_constraints/

Windows

Check the docker documentation to see, how you can use commands to set various parameters. https://docs.docker.com/docker-for-windows/

Building Docker image

• Clone the repo https://github.com/murphygroup/docker-cellorganizer-jupyter-notebook.

• Use the build script to create the docker image ( you will need the matlab binaries of CellOrganizer or it will download the latest from web)

• Once the image builds successfully ( docker images command can be used to check if image is created), you can go ahead to make the container.

• Use the run script to run the container with this docker image.

• The container starts with the url of accessing Jupyter notebook.

Visualizing Results

• To visualize a result, one can create a notebook cell (the notebook must reside in same location as index.html)

from IPython.core.display import HTML

HTML(filename=”./index.html”)

# Same folder location, ensures the embedded images don’t break while rendering.