Docker Layers
Going forward, IOx will support deployment of individual docker layers packaged using ioxclient. This way, every time an application is installed on a device, some or all of the available layers on the device that an application is dependent on will be reused, instead of copying them. This will greatly reduce the bandwidth required to deploy applications with common docker layers.
Below are some of the examples on how to generate and deploy applications using layering support in IOx.
Generate Individual Layers From Docker Image
To generate individual layers from a docker image use the below command.
dev@dev-VirtualBox:~/iox/ioxclient$ ioxclient docker package --layers
NAME:
package - Package an existing docker image as an IOx application
USAGE:
command package [command options] <docker_image> <project_dir>
DESCRIPTION:
To generate IOx layers from a docker image:
ioxclient docker package --layers <image_name> <project_dir>
To generate IOx layers from an IOx package:
ioxclient docker package --layers <iox_package> <project_dir>
OPTIONS:
--package-type, -p Use this option to specify the rootfs type: ext2 or ufs.
--auto, -a Use this flag to build IOx package according to labels in the docker image.This option has lower precedence over package-type option
--use-targz Use this to use gz compression on package
--skip-schema-validation Use this flag to skip package descriptor schema validation
--descriptor-merge, -m enable addition/override of information from docker inspect to package.yaml
--labels, -l Pass label(s) to be translated into IOx application descriptor fields.
--headroom, -r '0' For LXC type apps, specify the headroom (in MB) that needs to be created in the rootfs. Default is 5MB or 20% of rootfs size, whichever is greater
--layers, -s Use this flag to package docker image as IOx compatible layers
--rsa-key, -k Use this option to specify a RSA key in PEM format to sign the package
--certificate, -c Use this option to specify a x509 Certificate in PEM format
--skip-signing Use this option to skip package signing
dev@dev-VirtualBox:~/iox/alpine/pkg$ ioxclient docker package --layers alpine:3.3 ./
Currently active profile : default
Command Name: docker-package
Warning: package.yaml not present in project folder. Will attempt to generate one.
Retrieving docker image
Generating IOx Layers
Generating package metadata
Finding the minimum schema version for the descriptor file
Setting the descriptor schema version to 2.2
Validating generated descriptor file.
Validating descriptor file /tmp/desc052669482 with package schema definitions
Parsing descriptor file..
Found schema version 2.2
Loading schema file for version 2.2
Validating package descriptor file..
File /tmp/desc052669482 is valid under schema version 2.2
Parsing Package Metadata file : /home/dev/iox/alpine/pkg/.package.metadata
Wrote package metadata file : /home/dev/iox/alpine/pkg/.package.metadata
Generating IOx app package
No rsa key and/or certificate files to sign the package
Checking if package descriptor file is present..
Skipping descriptor schema validation..
Checking if package descriptor file is present..
Skipping descriptor schema validation..
Created Staging directory at : /tmp/300082412
Copying contents to staging directory
Checking for application runtime type
Couldn't detect application runtime type
Creating an inner envelope for application artifacts
Generated /tmp/300082412/artifacts.tar.gz
Calculating SHA1 checksum for package contents..
Parsing Package Metadata file : /tmp/300082412/.package.metadata
Wrote package metadata file : /tmp/300082412/.package.metadata
Root Directory : /tmp/300082412
Output file: /tmp/792682843
Path: .package.metadata
SHA1 : b5b32f07e95f21fb846af21789649408b8023f29
Path: artifacts.tar.gz
SHA1 : b1bf4c45a010b0d2f79b860f2dd393865e267436
Path: package.yaml
SHA1 : 425f4de862257214106e2fa70494078bd359a74a
Generated package manifest at package.mf
Generating IOx Package..
Package docker image alpine:3.3 at /home/dev/iox/alpine/pkg/package.tar, iox layers: iox_layers
Generated layers will be placed in a directory named "iox_layers" inside the project directory. If a need arises to modify the package.yaml or package_config.in files and repackage the application, use the "iox_layers" directory to create an IOx compatible package (ioxclient package).
An example to package an application using already generated layers.
dev@dev-VirtualBox:~/iox/ioxclient$ ioxclient package --layers
USAGE:
command package [command options] <path_to_dir>
OPTIONS:
--use-targz Use this to use gz compression on package
--skip-schema-validation Use this flag to skip package descriptor schema validation
--layers, -l Use this option to package application layers by specifying IOx layers directory
--rsa-key, -k Use this option to specify a RSA key in PEM format to sign the package
--certificate, -c Use this option to specify a x509 Certificate in PEM format
--skip-signing Use this option to skip package signing
dev@dev-VirtualBox:~/iox/alpine/pkg$ ls -l
total 8
drwxrwxr-x 2 dev dev 4096 Jul 24 11:57 iox_layers
-rwxr--r-- 1 dev dev 212 Jul 24 11:57 package.yaml
dev@dev-VirtualBox:~/iox/alpine/pkg$ ioxclient package --layers ./iox_layers/ ./
Currently active profile : default
Command Name: package
No rsa key and/or certificate files to sign the package
Checking if package descriptor file is present..
Validating descriptor file /home/dev/iox/alpine/pkg/package.yaml with package schema definitions
Parsing descriptor file..
Found schema version 2.2
Loading schema file for version 2.2
Validating package descriptor file..
File /home/dev/iox/alpine/pkg/package.yaml is valid under schema version 2.2
Checking if package descriptor file is present..
Skipping descriptor schema validation..
Created Staging directory at : /tmp/116629525
Copying contents to staging directory
Checking for application runtime type
Couldn't detect application runtime type
Creating an inner envelope for application artifacts
Generated /tmp/116629525/artifacts.tar.gz
Calculating SHA1 checksum for package contents..
Parsing Package Metadata file : /tmp/116629525/.package.metadata
Wrote package metadata file : /tmp/116629525/.package.metadata
Root Directory : /tmp/116629525
Output file: /tmp/626259568
Path: .package.metadata
SHA1 : 11f4783460e5faec6797e9e693d174192d5466fd
Path: artifacts.tar.gz
SHA1 : 299b79f918ed33a41dcfd3d91e3a11c1cee55af2
Path: package.yaml
SHA1 : 425f4de862257214106e2fa70494078bd359a74a
Generated package manifest at package.mf
Generating IOx Package..
Package generated at /home/dev/iox/alpine/pkg/package.tar
- The generated package does not the include actual layers. So, to install an application using the above generated package, all the required layers should already be deployed on the device.
- For a much easier way to install an application using layers, we recommend the "ioxclient application install --layers " command.
An example to generate layers from an existing IOx docker application package
dev@dev-VirtualBox:~/iox/alpine/pkg$ ioxclient docker package --layers ./package.tar ./
Currently active profile : caf
Command Name: docker-package
Extracting rootfs.tar from provided IOx package
Using the package descriptor file in the project dir
Validating descriptor file package.yaml with package schema definitions
Parsing descriptor file..
Found schema version 2.2
Loading schema file for version 2.2
Validating package descriptor file..
File package.yaml is valid under schema version 2.2
Generating IOx Layers
Generating package metadata
Parsing Package Metadata file : /home/dev/iox/alpine/pkg/.package.metadata
Wrote package metadata file : /home/dev/iox/alpine/pkg/.package.metadata
Generating IOx app package
No rsa key and/or certificate files to sign the package
Checking if package descriptor file is present..
Skipping descriptor schema validation..
Checking if package descriptor file is present..
Skipping descriptor schema validation..
Created Staging directory at : /tmp/893768253
Copying contents to staging directory
Checking for application runtime type
Couldn't detect application runtime type
Creating an inner envelope for application artifacts
Generated /tmp/893768253/artifacts.tar.gz
Calculating SHA1 checksum for package contents..
Parsing Package Metadata file : /tmp/893768253/.package.metadata
Wrote package metadata file : /tmp/893768253/.package.metadata
Root Directory : /tmp/893768253
Output file: /tmp/627339896
Path: .package.metadata
SHA1 : 38d3627bb74664642c3b2d90ea5a5008c8e7aa98
Path: artifacts.tar.gz
SHA1 : 2d7f919cd89912dcee072ef7a93589953b0cb86c
Path: package.yaml
SHA1 : 425f4de862257214106e2fa70494078bd359a74a
Generated package manifest at package.mf
Generating IOx Package..
Package docker image at /home/dev/iox/alpine/pkg/package.tar, iox layers: iox_layers
dev@dev-VirtualBox:~/iox/alpine/pkg$ ls -l
total 16
drwxrwxr-x 2 dev dev 4096 Jul 24 13:55 iox_layers
-rw-rw-r-- 1 dev dev 6144 Jul 24 13:55 package.tar
-rwxr--r-- 1 dev dev 212 Jul 24 13:55 package.yaml
- Note: Do not use the directory where the original package.tar is present as the project directory. The command will overwrite the original package.tar file.
Installing an Application From a Docker Image
To install an application directly from a docker image instead of going through packaging and installation steps, please look at the below example.
dev@dev-VirtualBox:~/Documents/iox/proj/src/ioxclient$ ioxclient application install --layers
USAGE:
command install [command options] <application_id> <archive or project dir> [docker_image]
DESCRIPTION:
if you want to install an application by using IOx layers, please look below for command usage.
ioxclient application install [--rsa-key<> --cert<>] --layer appid <project_dir> <docker_image>
OPTIONS:
--layers, -l Use this flag to install an application using IOx layers. When this option is specified all the other options are ignored
--resolve-dependencies, --rd Use this flag to activate automatic resolution of dependencies
--service-source, --src Use this flag to specify the source from which the dependent services will looked for
--service-activation-payload, -p Use this flag to pass activation payload for activating services
--rsa-key, -k Use this option to specify a RSA key in PEM format to sign the package
--certificate, -c Use this option to specify a x509 Certificate in PEM format
--skip-signing Use this option to skip package signing
dev@dev-VirtualBox:~/iox/alpine/pkg$ ioxclient application install --layers test ./ alpine:3.3
Currently active profile : caf
Command Name: application-install
Warning: package.yaml not present in project folder. Will attempt to generate one.
Retrieving docker image
Generating IOx Layers
Generating package metadata
Finding the minimum schema version for the descriptor file
Setting the descriptor schema version to 2.2
Validating generated descriptor file.
Validating descriptor file /tmp/desc367045569 with package schema definitions
Parsing descriptor file..
Found schema version 2.2
Loading schema file for version 2.2
Validating package descriptor file..
File /tmp/desc367045569 is valid under schema version 2.2
Parsing Package Metadata file : /home/dev/iox/alpine/pkg/.package.metadata
Wrote package metadata file : /home/dev/iox/alpine/pkg/.package.metadata
Generating IOx app package
No rsa key and/or certificate files to sign the package
Checking if package descriptor file is present..
Skipping descriptor schema validation..
Checking if package descriptor file is present..
Skipping descriptor schema validation..
Created Staging directory at : /tmp/005324187
Copying contents to staging directory
Checking for application runtime type
Couldn't detect application runtime type
Creating an inner envelope for application artifacts
Generated /tmp/005324187/artifacts.tar.gz
Calculating SHA1 checksum for package contents..
Parsing Package Metadata file : /tmp/005324187/.package.metadata
Wrote package metadata file : /tmp/005324187/.package.metadata
Root Directory : /tmp/005324187
Output file: /tmp/947273278
Path: .package.metadata
SHA1 : 0a31b6b204d9d3e99d009fed2d29114035fc0eee
Path: artifacts.tar.gz
SHA1 : 7bb3e4c09e02ee78e3759d1b6ef72532097817b7
Path: package.yaml
SHA1 : 425f4de862257214106e2fa70494078bd359a74a
Generated package manifest at package.mf
Generating IOx Package..
Package docker image alpine:3.3 at /home/dev/iox/alpine/pkg/package.tar, iox layers: iox_layers
Following layers are missing on the device: "9d2df2afed0b40cfecf86308306882a37f983aa3fd93e32ba352cbc5a731328f"
Installing layer iox_layers/9d2df2afed0b40cfecf86308306882a37f983aa3fd93e32ba352cbc5a731328f.tar.gz
Layer got added successfully.Successfully uploaded the layer 9d2df2afed0b40cfecf86308306882a37f983aa3fd93e32ba352cbc5a731328f
Installation Successful. App is available at : https://10.41.50.94:8443/iox/api/v2/hosting/apps/test
Successfully deployed
dev@dev-VirtualBox:~/Documents/iox/img/ac5/x86/alpine/pkg$
dev@dev-VirtualBox:~/Documents/iox/img/ac5/x86/alpine/pkg$ ioxclient application list
Currently active profile : caf
Command Name: application-list
List of installed App :
1. test ---> DEPLOYED
- Note: Using the above command, one can also deploy a signed package by providing the key and certificate using -k and -c options.
Manage Docker Applications' Layers
To manage the life cycle of docker applications' layers on the platform/device refer to the below command
dev@dev-VirtualBox:~$ ioxclient layer
NAME:
ioxclient layer - Manage life cycle of application layers
USAGE:
ioxclient layer command [command options] [arguments...]
COMMANDS:
list, l List deployed layers on the device
add, a add/deploy a particular layer to the device
delete, d delete a particular layer or all unused layers on the device
help, h Shows a list of commands or help for one command
OPTIONS:
--help, -h show help
--generate-bash-completion
Get a List of all the layers deployed
To get a list of all the layers currently deployed on the device, in JSON format, use the below command
dev@dev-VirtualBox:~$ ioxclient layer list
Currently active profile : 829
Command Name: layer-list
List of deployed layers:
{
"layers": [
{
"docker_id": "cf6fd3a072756da50619d3701c701f4a83792799fd61a24aeac0ac53302b9d28",
"layer_id": "45d5e45babebea2e70c7354be8a17ceda8a8db9e74b429e142a86d8697a59b2d",
"size": 42333184,
"symlinked_to": [
"test"
],
"used_by": [
"test"
]
},
{
"docker_id": "1a2e8a316114edb1b164dd645c4d49913c0b5cccf27142e026b7b0bdcb5f836b",
"layer_id": "cbede313b877303ac45010c4635432722f3db8c986ae3c082db1d0a916a1b18f",
"size": 1171968,
"symlinked_to": [
"test"
],
"used_by": [
"test"
]
},
{
"docker_id": "8dc8920290fc4566437bca6e5bedf637f902e79d5f0e603bdb7e3df121ceca6f",
"layer_id": "5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef",
"size": 1024,
"symlinked_to": [
"test"
],
"used_by": [
"test"
]
}
]
}
Add a Layer
To add a particular layer to the device use the below command
dev@dev-VirtualBox:~$ ioxclient layer add ./0ae878fde72e7076f919fb4c5f5ebff461f524365f870f970ac463eb4d047536.tar.gz
Currently active profile : 829
Command Name: layer-add
Layer got added successfully.
Successfully uploaded the layer 0ae878fde72e7076f919fb4c5f5ebff461f524365f870f970ac463eb4d047536
- Individual layers from a docker image can be generated using the command "ioxclient docker package". Please refer to the command's documentation for further details.
Delete a Layer
To delete a particular layer use the below command
dev@dev-VirtualBox:~$ ioxclient layer delete -h
NAME:
delete - delete a particular layer or all unused layers on the device
USAGE:
command delete [layer_id]
DESCRIPTION:
To delete a particular layer specify the layer id as an argument as shown below
ioxclient layer delete <layer_id>
To delete all unused layers on the device, use the below command
ioxclient layer delete
dev@dev-VirtualBox:~$ ioxclient layer delete 0ae878fde72e7076f919fb4c5f5ebff461f524365f870f970ac463eb4d047536
Currently active profile : 829
Command Name: layer-delete
Successfully removed the layer 0ae878fde72e7076f919fb4c5f5ebff461f524365f870f970ac463eb4d047536
dev@dev-VirtualBox:~$ ioxclient layer delete
Currently active profile : 829
Command Name: layer-delete
Delete all the unused layers on the device[y/n]:y
Moving forward with execution of the command
Successfully removed all unused layers
- Note: Layer id is the layer's file name excluding the tar.gz extension.