« Back to Technical Questions

RE: Inconsistent results installing package.

Combination View Flat View Tree View
Threads [ Previous | Next ]
Post New Thread
Inconsistent results installing package.
Answer
2/6/12 4:04 PM
We are upgrading to 1.6.2 and decided to completely repackage our application.  We do a clean install of 1.6.2 and add 1.6.2 app-dev.  Then we install our package.  Our package is built with YUM via the third_party_rpms_repository in our source directory.  What we have found is that between 1/3 and 1/2 the time the package doesn't have YUM.   We check this with find / -name yum and get no results.
Currently our workaround it to repeat the process from the beginning and so far it works on the second try. 
Unfortunately we have about 150 to upgrade and this isn't going to be realistic.
 
Where would I begin to look for why this is happening?
 
Thanks
Flag

RE: Inconsistent results installing package.
Answer
2/6/12 8:02 PM as a reply to TOM SCHAUFFLER.
Hi Tom,

It's quite strange that the yum installation is randomly failing. Are you installing your own user defined Linux OS or using the included CentOS?

Please try the steps below to debug the issue when yum has not been installed.

1. From the AXP prompt check the /var/log/messages.log file for error messages. Type: 'show log name messages.log interactive' and search for rpm errors.

2. In the guest shell of your application check the third_party_rpms_repository directory and make sure all the rpms are there.

3. In the guest shell of your application try to install the yum rpm and check if any errors occur. There will probably be a dependency error from which you can work backwards to determine the cause.

Thanks,

Brett
Flag

RE: Inconsistent results installing package.
Answer
2/9/12 9:28 PM as a reply to Brett Tiller.
Hi Tom,

It's quite strange that the yum installation is randomly failing. Are you installing your own user defined Linux OS or using the included CentOS?
 
We are using the included CentOS
 
Please try the steps below to debug the issue when yum has not been installed.

1. From the AXP prompt check the /var/log/messages.log file for error messages. Type: 'show log name messages.log interactive' and search for rpm errors.
 
This returned about 3 meg worth of text and the only result in searching for rpm was the following.
Feb  6 19:54:06 localhost err_handler: 2012 Feb  6 19:54:06 UTC +0000: err_handler: rpm-libs-4.4.2-48
 
Let me know if you want the logfile with the messages.log and how you want me to get it to you.
 
2. In the guest shell of your application check the third_party_rpms_repository directory and make sure all the rpms are there.
 
In the guest shell on a working and on a non working install I do not see any sign of the directory or the rpms. However on the working Yum and all of the other third party rpms are installed and working.  To me it appears as though on the failed install that the third party rpm directory is completely ignored during the install process.

3. In the guest shell of your application try to install the yum rpm and check if any errors occur. There will probably be a dependency error from which you can work backwards to determine the cause.

Thanks,

Brett
Flag

RE: Inconsistent results installing package.
Answer
2/9/12 11:53 PM as a reply to TOM SCHAUFFLER.
Hi Tom,

Looks like something may have occurred during the installation of the rpms. I'd like to check if the issue is occurring during packaging or install time. So in a case where the failure occurred so yum is not present in the guest OS, please send me the following docs:

A. The tech support logs from the AXP service module. You can get them by typing on the module: copy tech-support url ftp://<user>@<dir>/tech.tar.gz

B. The logs from the packaging of the your files. This will be located in a gzipped tar file in your package output directory alongside your packaged files.

You can post them via the steps below. Once you are done please let me know along with the file names.

1. Connect to ftp.cisco.com via anonymous ftp.
2. Files must be uploaded into a folder called 'incoming', which is not visible to anonymous users
If you are using a text-based ftp client, type the command "cd incoming"
If you are using a GUI ftp client such as FileZilla, enter 'incoming' manually as the destination for the remote site
3. Upload the file as normal. You will not be able to see it on the server, as commands to list the files in that folder have been disabled. You may see error messages related to that.

NOTE: You cannot upload a file with the same name more than once as this creates the potential for other users to overwrite the files you have uploaded. If you need to re-upload the same file, either use a new filename or ask the Cisco employee you are working with to remove the file from the server.


Thanks,

Brett
Flag

RE: Inconsistent results installing package.
Answer
2/21/12 9:37 PM as a reply to Brett Tiller.
Brett,
Sorry for the delay I ftp'd the files up as tech.tar and 20120130_3-55-25.log.tar on Feb 10, but replied to the email instead of posting back here.
Let me know if I need to ftp them up again as I know that gets cleared pretty frequently.

Thanks,

Tom
Flag

RE: Inconsistent results installing package.
Answer
2/21/12 10:20 PM as a reply to TOM SCHAUFFLER.
Tom,

Unfortunately I don't see the file on the server, so it must've been cleared. Please upload it again and let me know when it's ready.

Thanks,
Brett
Flag

RE: Inconsistent results installing package.
Answer
2/21/12 10:37 PM as a reply to Brett Tiller.
Files have been uploaded as tech.tar and 20120130_3-55-25.log.tar
Flag

RE: Inconsistent results installing package.
Answer
2/22/12 12:22 AM as a reply to TOM SCHAUFFLER.
Hi Tom,

I'm reviewing the files now. I don't see any reported issues during the packaging phase, however I have a few questions as I'm seeing a few unexpected error/warning messages during the installation of the third party rpms where the application installation of Yum fails.

1. Did you repackage the application using the 1.6.2 SDK?

2. When you used the previous release of AXP 1.5.3, did you experiencing any of the same failures when installing Yum?

3. You mentioned that as a work around reinstalling appears to resolve the issue. What are your complete workaround steps?

Thanks,

Brett
Flag

RE: Inconsistent results installing package.
Answer
2/22/12 12:38 AM as a reply to Brett Tiller.
Hi Tom,

Please ignore my first question. The logs show that you had repackaged the application using the 1.6.2 SDK.

Thanks,
Brett
Flag

RE: Inconsistent results installing package.
Answer
2/22/12 2:58 AM as a reply to Brett Tiller.
Hi Tom,

We haven't had any luck so far reproducing the issue you are experiencing. Would you mind sharing your application package and payload files, .pkg and .prt1 with us? This way we can review them and test them in our lab. If you're ok with this, please upload them to the file server and let me know the name(s) when done.

Thanks,
Brett
Flag

RE: Inconsistent results installing package.
Answer
2/22/12 2:36 PM as a reply to Brett Tiller.
2. We did not experience this until 1.6.2. Both versions of our package perform the same function, the only difference is that when we upgraded the OS on the NME's we repackaged the application rpms.

3. As soon as we have command line access to the application we test to see if yum is installed, if it is not then none of our third party rpms are installed. At this point we start all over from the initial software install clean of the 1.6.2 software and redo the build entirely.

files uploaded as :
wds.2.0.0.pkg
wds.2.0.0.prt1
wds axp.txt - step by step instructions on how we install and setup the app.
Flag

RE: Inconsistent results installing package.
Answer
2/23/12 2:46 AM as a reply to TOM SCHAUFFLER.
Tom,

Thanks for the files. I've run several tests so far following your steps, but have not yet reproduced the issue you are experiencing. I will continue investigating this issue and get back to you when I have more data to provide.

Thanks,

Brett
Flag

RE: Inconsistent results installing package.
Answer
2/23/12 10:56 PM as a reply to Brett Tiller.
Tom,

I've run five separate tests to attempt to reproduce the issue and have not been able to do so. However, I have also been researching your issue based upon the messages in the logs you've provided. Next time you encounter this issue, I'd like you to try the commands below and let me know if it resolves it. Based upon my research I think the rpm files were probably installed; however, the rpm database was somehow corrupted, so deleting the database files and rebuilding the database might resolve the issue.

1. Access the wds guest environment shell.

2. type: rm -f /var/lib/rpm/__db.*

3. type: rpm --rebuilddb

4. type: yum


Thanks,

Brett
Flag

RE: Inconsistent results installing package.
Answer
2/24/12 4:44 PM as a reply to Brett Tiller.
Here are the results

bash-3.2# rm -f /var/lib/rpm/_db.*
bash-3.2# rpm --rebuilddb
bash-3.2# yum
bash: yum: command not found
bash-3.2#
Flag

RE: Inconsistent results installing package.
Answer
2/27/12 8:59 PM as a reply to TOM SCHAUFFLER.
Tom,

Before I discuss the rpm installation failure issue I just wanted to mention that you can bundle everything together with the AXP platform and do one installation which would be a huge timesaver in your case. The script that bundles packages together is located in the sdk tools directory and is called axp_bundle.sh. You can read more about bundling in the AXP Developer Guide section 'Bundling Tool'.

The install.log and messages.log files are showing errors in the rpm database during the install of each rpm. Since we cannot reproduce that error in our labs, I'm going to now assume that this error is causing the installation of each rpm to fail. As I mentioned earlier via research I've done removing and then rebuilding the rpm database should resolve that issue. As a result I would suggest writing a script that will install the rpms and check for errors after each install and if one has occurred rebuild the rpm database and then reattempt the installation. The script would therefore manage the rpm installation rather than the AXP host. I've written such a script below as an example that would be added to or replace the post-install.sh script that gets called when the installation has been completed. Note that the rpm files are now located in the directory called "third_party_rpms" since that name directory change will cause the AXP host to not attempt to install the rpms itself. Although this script is merely an example, and is not tested, you're certainly welcome to use, modify and of course test it for your own needs.

#! /bin/sh

ln -s /bin/bash /bin/console

theDir="third_party_rpms"
rpms="python-urlgrabber-3.1.0-2.noarch.rpm libtalloc-2.0.1-11.el5.i386.rpm libtiff-3.8.2-7.el5_6.7.i386.rpm samba3x-3.5.4-0.83.el5.i386.rpm python-sqlite-1.1.7-1.2.1.i386.rpm libtdb-1.2.1-6.el5.i386.rpm iproute-2.6.18-7.el5.i386.rpm tftp-server-0.49-2.el5.centos.i386.rpm logrotate-3.7.4-12.i386.rpm ethtool-5-1.el5.i386.rpm mcstrans-0.2.7-1.el5.i386.rpm module-init-tools-3.3-0.pre3.1.37.el5.i386.rpm yum-metadata-parser-1.1.2-2.el5.i386.rpm tcp_wrappers-7.6-40.7.el5.i386.rpm libpng-1.2.10-7.1.el5_5.3.i386.rpm samba3x-winbind-3.5.4-0.83.el5.i386.rpm perl-5.8.8-32.el5_6.3.i386.rpm libsysfs-2.0.0-6.i386.rpm expat-1.95.8-8.2.1.i386.rpm mingetty-1.07-5.2.2.i386.rpm MAKEDEV-3.23-1.2.i386.rpm zlib-1.2.3-3.i386.rpm glibc-common-2.5-24.i386.rpm libvolume_id-095-14.16.el5.i386.rpm psmisc-22.2-6.i386.rpm gdbm-1.8.0-26.2.1.i386.rpm vixie-cron-4.1-77.el5_4.1.i386.rpm libsmbclient-3.0.33-3.29.el5_6.2.i386.rpm centos-release-5-2.el5.centos.i386.rpm samba3x-client-3.5.4-0.83.el5.i386.rpm libxml2-2.6.26-2.1.2.1.i386.rpm libjpeg-6b-37.i386.rpm libgpg-error-1.4-2.i386.rpm xinetd-2.3.14-13.el5.i386.rpm db4-4.3.29-9.fc6.i386.rpm samba3x-common-3.5.4-0.83.el5.i386.rpm udev-095-14.16.el5.i386.rpm rpm-python-4.4.2-48.el5.i386.rpm rsync-3.0.6-4.el5.i386.rpm gnutls-1.4.1-3.el5_4.8.i386.rpm binutils-2.17.50.0.6-6.el5.i386.rpm cups-libs-1.3.7-26.el5_6.1.i386.rpm yum-3.2.8-9.el5.centos.1.noarch.rpm openldap-2.3.43-12.el5_6.7.i386.rpm libgcrypt-1.4.4-5.el5.i386.rpm python-2.4.3-21.el5.i386.rpm python-elementtree-1.2.6-5.i386.rpm cyrus-sasl-lib-2.1.22-5.el5_4.3.i386.rpm"

for theRpm in $rpms
do
rpm -ivh --nodeps /$theDir/$theRpm
if [ $? ne 0 ]; then
rm -f /var/lib/rpm/_db.*
rpm --rebuilddb
rpm -ivh --nodeps /$theDir/$theRpm
fi

done
Flag