Overview
- This page describes how to use the docker tooling to develop a python application which is using the python modules containing C/C++ components.
- The application uses the python module 'psutils' to retrieve the information about the currently running processes.
- It displays the information about the active processes in the container - pid, start time and status - in a tabular format.
Goals
- Demonstrate how to install/build a platform dependent python module (module which contains C/C++ component) using pip in natively running docker container.
- Show how to copy built python module and create python runtime environment in base docker image for the python application
Sample Code
This sample application code is maintained here.
Clone this repo and use branch master. Find the application under directory 'python-c-app'
$ git clone https://github.com/CiscoIOx/docker-demo-apps.git -b master
$ cd docker-demo-apps/python-c-app
Some parts of the code may be reproduced in this document to give more context. But always use the above git repo and branch for the latest code.
Procedure Overview
- Implementing a python application which is using platform dependent python module
- Creating a docker image to setup the build environment for installing/building python module
- Run a docker container from the image created in previous step and build the python module
- Creating a docker image with application, built module and its required runtime environment
- Testing the application locally using docker tools
- Creating an IOx application package from the docker image
- Deploying and testing on the target platform
Procedure
This section describes above steps with more details.
Before going through the tutorial, get familiar with cisco provided artifacts for an IOx app development.
Note that the application examples are demonstrated for IR800 platform, but the same procedure will be applicable for all other supported IOx platforms. Developer needs to use the right docker base image as per the IOx platform. Refer for images supported platforms & corresponding base images
The workspace (directory structure) for this application is looks as shown below
$ tree
.
├── app
│ ├── python-c-app.py
│ └── python_modules
├── dev
│ └── Dockerfile
└── Dockerfile
4 directories, 4 files
Here, we have used 2 Dockerfiles for developing the python application:
- Dockerfile under python-c-app/dev/ pulls a docker base image and installs the toolchain, python pip and python development package. This is used for building a platform dependent python module.
- Dockerfile under python-c-app pulls a docker base image, copies the python application, built module and installs runtime required packages for the application.
Implementing a python application
Write an application which prints the information about the active processes in the container - pid, start time and status - in a tabular format.
Creating a project directory
$ mkdir python-c-app
$ cd python-c-app
Writing a python application
Here is the python code:
$ mkdir app
$ cd app
$ vi python-c-app.py
import datetime
import psutil
# This program displays the current running processes in tabular form
def main():
""" Prints current active processes """
# Create a format and print table headings
row_format = "{:<40} {:<10} {:<30} {:<20}"
print row_format.format("Proc Name", "Proc ID", "Start Time", "Status",
"Priority")
print "-"*100
# Iterate over all running processes
for proc in psutil.process_iter():
# Get process create time in the standard time format
time = datetime.datetime.fromtimestamp(proc.create_time()).strftime(
"%Y-%m-%d %H:%M:%S")
# Print process's name, pid, create time and status in a table format
print row_format.format(proc.name(),
proc.pid,
time,
proc.status())
if __name__ == '__main__':
main()
Writing a loop application
- Docker type applications are not based on /sbin/init to make the container alive and to start the init scripts.
- We need to use some loop application which will run in the background to make the container alive.
- This will be used in a Dockerfile as an application entry point. You will understand more details in the subsequent sections how to use this loop application.
Here is a simple loop application:
$ vi loop.sh
#!/bin/sh
while [ 1 ]; do
sleep 1000
done
Note: Use this method only if you want to keep the container alive. If it is not required, skip this.
Creating a docker image to build the python module
Write a Dockerfile
This Dockerfile does the following tasks:
- Pulls the docker base image
- Install the build environment for C/C++
- Install the python pip and python development packages
- Build the 'psutil' module using pip
Here is a Dockerfile:
$ cd python-c-app/dev
$ vi Dockerfile
FROM devhub-docker.cisco.com/iox-docker/ir800/base-rootfs
RUN opkg update
RUN opkg install iox-toolchain
RUN opkg install python-pip
RUN opkg install python-misc
RUN opkg install python-dev
RUN pip install psutil
Creating a docker image
Login to the devhub using docker daemon before building a docker image. Instructions for devhub login
Now let us build an image from this Dockerfile
$ docker build -t python-dev .
Check if the image is successfully created using following command:
$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
python-dev latest 0198f2056b51 19 minutes ago 210.6 MB
Run the image locally to test the built python module
So far we have built the python module 'psutil' and it is contain in the new image python-dev. Let us run that image locally to test the built python module.
At this stage we are sharing host volume with the container to copy the built python module to the host.
```
$ cd python-c-app/
$ ls
app dev Dockerfile
$ docker run -v pwd/app:/opt/share -it python-dev /bin/sh
sh-4.3# cd opt/share/
sh-4.3# ls
python-c-app.py python_modules
sh-4.3# ./python-c-app.py
Proc Name Proc ID Start Time Status
sh 1 2016-10-28 06:03:49 sleeping python 8 2016-10-28 06:03:57 running sh-4.3# sh-4.3# cp /usr/lib/python2.7/site-packages/psutil psutil/ psutil-4.4.2-py2.7.egg-info/ sh-4.3# cp -r /usr/lib/python2.7/site-packages/psutil* python_modules/ sh-4.3# sh-4.3# exit exit
**Note**: Python modules 'psutil' installs under '/usr/lib/python2.7/site-packages/',
hence we copied the module from that location to the shared volume.
#### Creating a docker image with the python application and python runtime support
Now we have the python application and the required python module 'psutil' built for the target. Next step is to create a docker
image with the python application and python runtime support.
##### Write a Dockerfile
This Dockerfile does the following tasks:
* Pulls the docker base image
* Copies the python application and built module
* Install python runtime packages
* Specifies the application entry point
Here is a Dockerfile content:
$ ls app dev Dockerfile $ cat Dockerfile FROM devhub-docker.cisco.com/iox-docker/ir800/base-rootfs
RUN opkg update RUN opkg install python RUN opkg install python-modules RUN opkg install python-misc COPY app/python_modules/ /usr/lib/python2.7/site-packages/ COPY app/python-c-app.py /opt/app/ COPY app/loop.sh /opt/app/ CMD ["/opt/app/loop.sh"]
Note: In this example, all python modules are installed. Developer can cherry pick only the application required python modules
to optimize the application package size.
##### Create a docker image
$ docker build -t python-app . Sending build context to Docker daemon 1.176 MB Sending build context to Docker daemon Step 0 : FROM devhub-docker.cisco.com/iox-docker/ir800/base-rootfs ---> 7b091cf2423f Step 1 : RUN opkg update . . . Step 8 : CMD /opt/apps/loop.sh ---> Running in ba6a18daebe5 ---> 81e797509c9a Removing intermediate container ba6a18daebe5 Successfully built 81e797509c9a
Note: In above code snippet, the docker build command generates lot of console messages. Only partial output is shown.
##### Test the application locally using docker tools
* Now the docker image created in above step has application, 'psutil' module and its required runtime environment.
Developer can start docker container from that image and test the application locally.
* Following code snippet shows how to start container locally and test the application.
$ docker run -it python-app /bin/sh / # cd opt/app/ /opt/app # ls loop.sh python-c-app.py /opt/app # ./python-c-app.py Proc Name Proc ID Start Time Status
sh 1 2016-10-28 17:13:53 sleeping python 14 2016-10-28 17:14:00 running /opt/app #
#### Create an IOx package and deploy to the target device
* Once the final docker image is ready with the application and its required runtime environment/dependencies,
developer can create an IOx package from that image.
* ioxclient utility can be used to deploy the IOx application package and to manage the application lifecycle.
Refer this page for ioxclient usage: [ioxclient](../ioxclient/ioxclient-reference.md)
##### Package descriptor file (.yaml file)
Package descriptor file specifies requirements for the application. For docker type applications, **creating a package
descriptor file is optional**. ioxclient creates a default package descriptor file while creating IOx docker
application package. If the developer want to add custom entries in a package descriptor file, he can add those
entries in Dockerfile using docker's LABEL directive. Ioxclient uses those labels, add it in a package descriptor
file during creation of an IOx docker application package.
Refer **[ioxclient](../ioxclient/ioxclient-reference.md)** page to learn how to use docker Label's
and package.yaml.
Here is a sample application descriptor (package.yaml) file looks like:
**Note:** If you are using application descriptor file for non x86 architecture, Refer the **[docker images table](./docker-hub.md)** for appropriate value of "cpuarch" in yaml file
Ex: For ie4k platform use the 'cpuarch' as 'ppc' in following yaml file
$ cat package.yaml descriptor-schema-version: "2.2"
info: name: PythonCapp description: "Python application with platform dependent python module" version: "1.0" author-link: "http://www.cisco.com" author-name: "Cisco Systems"
app:
Indicate app type (vm, paas, lxc etc.,)
cpuarch: "x86_64" type: docker resources: profile: c1.small
network:
-
interface-name: eth0
ports:
tcp: [8000]
udp: [10000]
Specify runtime and startup
startup: rootfs: rootfs.tar target: ["/opt/app/loop.sh"]
##### Create an IOx application package
* Use the ioxclient (>= 1.4.0) as shown below to create an IOx package from docker image python-app.
* Refer this page for ioxclient usage: [ioxclient](../ioxclient/ioxclient-reference.md)
* Create an empty directory ex. 'app_package' and execute the ioxclient command in this directory.
* Make sure the profile for target platform is created and activated before creating package
$ cd python-cpp-app/ $ mkdir app_package $ cd app_package/ $ ioxclient docker package -a python-app . Currently active profile : default Command Name: docker-package No package type specified, but auto flag is set cpu arch is x86, generating docker app x86_64 Generating docker style app The Image is better left in it's pristine state Warning: package.yaml not present in project folder. Will attempt to generate one. Generation complete. Validating generated descriptor file.
Validating descriptor file /tmp/desc651221242 with package schema definitions
Parsing descriptor file..
Found schema version 2.2
Loading schema file for version 2.2
Validating package descriptor file..
File /tmp/desc651221242 is valid under schema version 2.2
Package MetaData file was not found at /home/
* It creates an IOx application package at 'python-c-app/app_package/package.tar'
* The package.tar is an IOx application package that can be deployed on an IOx platform
##### Deploy and start the IOx application
* It is required to create an ioxclient profile for your target device and activate it before follow the next steps.
* Below is shown how to deploy, activate and start the application.
$ ioxclient profiles activate default Active Profile : default Activating Profile default Saving current configuration
$ ioxclient app install python_app ./app_package/package.tar Currently active profile : default Command Name: application-install Installation Successful. App is available at : https://1.100.30.65:8443/iox/api/v2/hosting/apps/python_app Successfully deployed
$ ioxclient app activate python_app Currently active profile : default Command Name: application-activate App python_app is Activated
$ ioxclient app start python_app Currently active profile : default Command Name: application-start App python_app is Started
$ ioxclient app console python_app Currently active profile : default Command Name: application-console Console setup is complete.. Running command : [ssh -p 2022 -i python_app.pem appconsole@1.100.30.65] SSH client is not present on the platform. Attempting to connect using native SSH implementation..., press Ctrl+C to exit
Connecting to appconsole@1.100.30.65:2022 using pem file python_app.pem / # cd opt/app /opt/app # ls loop.sh python-c-app.py /opt/app # ./python-c-app.py Proc Name Proc ID Start Time Status
startcontainer. 1 2016-10-28 18:56:01 sleeping loop.sh 15 2016-10-28 18:56:08 sleeping sleep 16 2016-10-28 18:56:08 sleeping sh 17 2016-10-28 18:56:10 sleeping python 19 2016-10-28 18:56:24 running /opt/app #