Overview

• This page describes how to use the docker tooling to develop a python application which is using the python modules containing C/C++ components.
• The application uses the python module 'psutils' to retrieve the information about the currently running processes.
• It displays the information about the active processes in the container - pid, start time and status - in a tabular format.

Goals

• Demonstrate how to install/build a platform dependent python module (module which contains C/C++ component) using pip in natively running docker container.
• Show how to copy built python module and create python runtime environment in base docker image for the python application

Sample Code

This sample application code is maintained here.

Clone this repo and use branch master. Find the application under directory 'python-c-app'

Copy$ git clone https://github.com/CiscoIOx/docker-demo-apps.git -b master
$ cd docker-demo-apps/python-c-app

Some parts of the code may be reproduced in this document to give more context. But always use the above git repo and branch for the latest code.

Procedure Overview

• Implementing a python application which is using platform dependent python module
• Creating a docker image to setup the build environment for installing/building python module
• Run a docker container from the image created in previous step and build the python module
• Creating a docker image with application, built module and its required runtime environment
• Testing the application locally using docker tools
• Creating an IOx application package from the docker image
• Deploying and testing on the target platform

Procedure

This section describes above steps with more details.

Before going through the tutorial, get familiar with cisco provided artifacts for an IOx app development.

Note that the application examples are demonstrated for IR800 platform, but the same procedure will be applicable for all other supported IOx platforms. Developer needs to use the right docker base image as per the IOx platform. Refer for images supported platforms & corresponding base images

The workspace (directory structure) for this application is looks as shown below

Copy$ tree
.
├── app
│   ├── python-c-app.py
│   └── python_modules
├── dev
│   └── Dockerfile
└── Dockerfile

4 directories, 4 files

Here, we have used 2 Dockerfiles for developing the python application:

• Dockerfile under python-c-app/dev/ pulls a docker base image and installs the toolchain, python pip and python development package. This is used for building a platform dependent python module.
• Dockerfile under python-c-app pulls a docker base image, copies the python application, built module and installs runtime required packages for the application.

Implementing a python application

Write an application which prints the information about the active processes in the container - pid, start time and status - in a tabular format.

Creating a project directory

Copy$ mkdir python-c-app
$ cd python-c-app

Writing a python application

Here is the python code:

Copy$ mkdir app
$ cd app
$ vi python-c-app.py


import datetime
import psutil


# This program displays the current running processes in tabular form
def main():
    """ Prints current active processes """
    # Create a format and print table headings
    row_format = "{:<40} {:<10} {:<30} {:<20}"
    print row_format.format("Proc Name", "Proc ID", "Start Time", "Status",
                            "Priority")
    print "-"*100
    # Iterate over all running processes
    for proc in psutil.process_iter():
        # Get process create time in the standard time format
        time = datetime.datetime.fromtimestamp(proc.create_time()).strftime(
            "%Y-%m-%d %H:%M:%S")
        # Print process's name, pid, create time and status in a table format
        print row_format.format(proc.name(),
                                proc.pid,
                                time,
                                proc.status())

if __name__ == '__main__':
    main()

Writing a loop application

• Docker type applications are not based on /sbin/init to make the container alive and to start the init scripts.
• We need to use some loop application which will run in the background to make the container alive.
• This will be used in a Dockerfile as an application entry point. You will understand more details in the subsequent sections how to use this loop application.

Here is a simple loop application:

Copy$ vi loop.sh
#!/bin/sh

while [ 1 ]; do
            sleep 1000
done

Note: Use this method only if you want to keep the container alive. If it is not required, skip this.

Creating a docker image to build the python module

Write a Dockerfile

This Dockerfile does the following tasks:

• Pulls the docker base image
• Install the build environment for C/C++
• Install the python pip and python development packages
• Build the 'psutil' module using pip

Here is a Dockerfile:

Copy$ cd python-c-app/dev
$ vi Dockerfile
FROM devhub-docker.cisco.com/iox-docker/ir800/base-rootfs

RUN opkg update
RUN opkg install iox-toolchain
RUN opkg install python-pip
RUN opkg install python-misc
RUN opkg install python-dev
RUN pip install psutil

Creating a docker image

Login to the devhub using docker daemon before building a docker image. Instructions for devhub login

Now let us build an image from this Dockerfile

Copy$ docker build -t python-dev . 

Check if the image is successfully created using following command:

Copy$ docker images
REPOSITORY                                                 TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
python-dev                                                 latest              0198f2056b51        19 minutes ago      210.6 MB

Run the image locally to test the built python module

So far we have built the python module 'psutil' and it is contain in the new image python-dev. Let us run that image locally to test the built python module.

At this stage we are sharing host volume with the container to copy the built python module to the host. ``` $ cd python-c-app/ $ ls app dev Dockerfile $ docker run -v pwd/app:/opt/share -it python-dev /bin/sh sh-4.3# cd opt/share/ sh-4.3# ls python-c-app.py python_modules sh-4.3# ./python-c-app.py Proc Name Proc ID Start Time Status

sh 1 2016-10-28 06:03:49 sleeping python 8 2016-10-28 06:03:57 running sh-4.3# sh-4.3# cp /usr/lib/python2.7/site-packages/psutil psutil/ psutil-4.4.2-py2.7.egg-info/ sh-4.3# cp -r /usr/lib/python2.7/site-packages/psutil* python_modules/ sh-4.3# sh-4.3# exit exit

Copy**Note**: Python modules 'psutil' installs under '/usr/lib/python2.7/site-packages/', 
hence we copied the module from that location to the shared volume.

#### Creating a docker image with the python application and python runtime support

Now we have the python application and the required python module 'psutil' built for the target. Next step is to create a docker 
image with the python application and python runtime support.

##### Write a Dockerfile
This Dockerfile does the following tasks:

* Pulls the docker base image
* Copies the python application and built module
* Install python runtime packages
* Specifies the application entry point

Here is a Dockerfile content:

$ ls app dev Dockerfile $ cat Dockerfile FROM devhub-docker.cisco.com/iox-docker/ir800/base-rootfs

RUN opkg update RUN opkg install python RUN opkg install python-modules RUN opkg install python-misc COPY app/python_modules/ /usr/lib/python2.7/site-packages/ COPY app/python-c-app.py /opt/app/ COPY app/loop.sh /opt/app/ CMD ["/opt/app/loop.sh"]

CopyNote: In this example, all python modules are installed. Developer can cherry pick only the application required python modules
to optimize the application package size.
##### Create a docker image

$ docker build -t python-app . Sending build context to Docker daemon 1.176 MB Sending build context to Docker daemon Step 0 : FROM devhub-docker.cisco.com/iox-docker/ir800/base-rootfs ---> 7b091cf2423f Step 1 : RUN opkg update . . . Step 8 : CMD /opt/apps/loop.sh ---> Running in ba6a18daebe5 ---> 81e797509c9a Removing intermediate container ba6a18daebe5 Successfully built 81e797509c9a

CopyNote: In above code snippet, the docker build command generates lot of console messages. Only partial output is shown.

##### Test the application locally using docker tools
* Now the docker image created in above step has application, 'psutil' module and its required runtime environment.
Developer can start docker container from that image and test the application locally.
* Following code snippet shows how to start container locally and test the application.

$ docker run -it python-app /bin/sh / # cd opt/app/ /opt/app # ls loop.sh python-c-app.py /opt/app # ./python-c-app.py Proc Name Proc ID Start Time Status

sh 1 2016-10-28 17:13:53 sleeping python 14 2016-10-28 17:14:00 running /opt/app #

Copy
#### Create an IOx package and deploy to the target device
* Once the final docker image is ready with the application and its required runtime environment/dependencies,
developer can create an IOx package from that image.
* ioxclient utility can be used to deploy the IOx application package and to manage the application lifecycle.

Refer this page for ioxclient usage: [ioxclient](../ioxclient/ioxclient-reference.md)

##### Package descriptor file (.yaml file)
Package descriptor file specifies requirements for the application. For docker type applications, **creating a package
descriptor file is optional**. ioxclient creates a default package descriptor file while creating IOx docker
application package. If the developer want to add custom entries in a package descriptor file, he can add those
entries in Dockerfile using docker's LABEL directive. Ioxclient uses those labels, add it in a package descriptor
file during creation of an IOx docker application package.

Refer **[ioxclient](../ioxclient/ioxclient-reference.md)** page to learn how to use docker Label's
and package.yaml.

Here is a sample application descriptor (package.yaml) file looks like:

**Note:** If you are using application descriptor file for non x86 architecture, Refer the **[docker images table](./docker-hub.md)** for appropriate value of "cpuarch" in yaml file

Ex: For ie4k platform use the 'cpuarch' as 'ppc' in following yaml file

$ cat package.yaml descriptor-schema-version: "2.2"

info: name: PythonCapp description: "Python application with platform dependent python module" version: "1.0" author-link: "http://www.cisco.com" author-name: "Cisco Systems"

app:

Indicate app type (vm, paas, lxc etc.,)

cpuarch: "x86_64" type: docker resources: profile: c1.small

Copynetwork:
  -
    interface-name: eth0
    ports:
        tcp: [8000]
        udp: [10000]

Specify runtime and startup

startup: rootfs: rootfs.tar target: ["/opt/app/loop.sh"]

Copy
##### Create an IOx application package
* Use the ioxclient (>= 1.4.0) as shown below to create an IOx package from docker image python-app.
* Refer this page for ioxclient usage: [ioxclient](../ioxclient/ioxclient-reference.md)
* Create an empty directory ex. 'app_package' and execute the ioxclient command in this directory. 
* Make sure the profile for target platform is created and activated before creating package

$ cd python-cpp-app/ $ mkdir app_package $ cd app_package/ $ ioxclient docker package -a python-app . Currently active profile : default Command Name: docker-package No package type specified, but auto flag is set cpu arch is x86, generating docker app x86_64 Generating docker style app The Image is better left in it's pristine state Warning: package.yaml not present in project folder. Will attempt to generate one. Generation complete. Validating generated descriptor file.

Validating descriptor file /tmp/desc651221242 with package schema definitions Parsing descriptor file.. Found schema version 2.2 Loading schema file for version 2.2 Validating package descriptor file.. File /tmp/desc651221242 is valid under schema version 2.2 Package MetaData file was not found at /home//projects/test_sde/python-cpp-app/app_package/.package.metadata Wrote package metadata file : /home//projects/test_sde/python-cpp-app/app_package/.package.metadata Checking if package descriptor file is present.. Validating descriptor file /home//projects/test_sde/python-cpp-app/app_package/package.yaml with package schema definitions Parsing descriptor file.. Found schema version 2.2 Loading schema file for version 2.2 Validating package descriptor file.. File /home//projects/test_sde/python-cpp-app/app_package/package.yaml is valid under schema version 2.2 Created Staging directory at : /tmp/615645201 Copying contents to staging directory Checking for application runtime type Couldn't detect application runtime type Creating an inner envelope for application artifacts Generated /tmp/615645201/artifacts.tar.gz Calculating SHA1 checksum for package contents.. Parsing Package Metadata file : /tmp/615645201/.package.metadata Wrote package metadata file : /tmp/615645201/.package.metadata Root Directory : /tmp/615645201 Output file: /tmp/789110844 Path: .package.metadata SHA1 : 98b31ec7867f810671f792f00fc0319776b2c140 Path: artifacts.tar.gz SHA1 : f2b717837753a6cfe925d1c46a73fcd64763a6a2 Path: package.yaml SHA1 : 399233104067f5b6ca6687364106d30f46af77d0 Generated package manifest at package.mf Generating IOx Package.. Package docker image python-app at /home//projects/test_sde/python-cpp-app/app_package/package.tar

Copy* It creates an IOx application package at 'python-c-app/app_package/package.tar'
* The package.tar is an IOx application package that can be deployed on an IOx platform

##### Deploy and start the IOx application
* It is required to create an ioxclient profile for your target device and activate it before follow the next steps.
* Below is shown how to deploy, activate and start the application.

$ ioxclient profiles activate default Active Profile : default Activating Profile default Saving current configuration

$ ioxclient app install python_app ./app_package/package.tar Currently active profile : default Command Name: application-install Installation Successful. App is available at : https://1.100.30.65:8443/iox/api/v2/hosting/apps/python_app Successfully deployed

$ ioxclient app activate python_app Currently active profile : default Command Name: application-activate App python_app is Activated

$ ioxclient app start python_app Currently active profile : default Command Name: application-start App python_app is Started

$ ioxclient app console python_app Currently active profile : default Command Name: application-console Console setup is complete.. Running command : [ssh -p 2022 -i python_app.pem appconsole@1.100.30.65] SSH client is not present on the platform. Attempting to connect using native SSH implementation..., press Ctrl+C to exit

Connecting to appconsole@1.100.30.65:2022 using pem file python_app.pem / # cd opt/app /opt/app # ls loop.sh python-c-app.py /opt/app # ./python-c-app.py Proc Name Proc ID Start Time Status

startcontainer. 1 2016-10-28 18:56:01 sleeping loop.sh 15 2016-10-28 18:56:08 sleeping sleep 16 2016-10-28 18:56:08 sleeping sh 17 2016-10-28 18:56:10 sleeping python 19 2016-10-28 18:56:24 running /opt/app #

Copy