« Back to CVP - All Versions

RE: New Message from Hemal Mehta in Customer Voice Portal (CVP) - CVP - All

Combination View Flat View Tree View
Threads [ Previous | Next ]
Hey guys,
 
We use the outbound app (.tcl polling) from our gateways to make outbound calls.  The vxml browsers seem to run fine for a few weeks and then they refuse to make any more outbound calls even when we have data in the database.  The only way to fix the issue is to reload the browser.  I really want to figure this out.  Probably an IOS bug that's not documented yet.
 
Anyway - in the logs on the browser we can see this:
 
Apr 13 07:46:32.156 CDT: //0/872DACE586A1/VXML:/vxml_block_build: 
   CALL_ERROR; http://removed:7000/CVP/Server
   at line 49: block name _in102507 is not unique
Apr 13 07:46:32.156 CDT: //0/872DACE586A1/VXML:/vxml_parse: 
   
Apr 13 07:46:32.156 CDT: vxml_parse: XML_Parse success err=0
Apr 13 07:46:32.156 CDT: vxml_tree_delete_for_parse_failure:mem_mgr_mempool_free: mempool=NULL
Apr 13 07:46:32.156 CDT: //0/872DACE586A1/VXML:/vxml_session_delete: 
    
Apr 13 07:46:32.156 CDT: vxml_session_delete:mem_mgr_mempool_free: mem_refcnt(6809FAE0)=0 - mempool cleanup
Apr 13 07:46:32.156 CDT: vxml_session_delete:mem_mgr_mempool_free: mempool=NULL
Apr 13 07:46:32.156 CDT: //755615/872DACE586A1/VXML:/vxml_stop_fetchaudio_load_fail: 
    
Apr 13 07:46:32.156 CDT: //755615/872DACE586A1/VXML:/vxml_bgload_post_done: 
   CALL_ERROR; http://removed:7000/CVP/Server
   loaded document (http://removed:7000/CVP/Server) parse error
Apr 13 07:46:32.156 CDT: //755615/872DACE586A1/VXML:/vxml_event_proc: 
            <event>: event=error.badfetch status=0

In the activity logs we see this:


10.173.90.34.1334320574249.1078589.OBND05,04/13/2012 07:36:15.171,Makecall_01,custom,DNIS,(removed)10.173.90.34.1334320574249.1078589.OBND05,04/13/2012 07:36:15.175,Makecall_01,interaction,audio_group,initial_audio_group
10.173.90.34.1334320574249.1078589.OBND05,04/13/2012 07:36:15.587,Makecall_01,interaction,noinput,1
10.173.90.34.1334320574249.1078589.OBND05,04/13/2012 07:36:15.587,Makecall_01,interaction,audio_group,noinput_audio_group
10.173.90.34.1334320574249.1078589.OBND05,04/13/2012 07:36:15.359,Makecall_01,element,error,error.badfetch
10.173.90.34.1334320574249.1078589.OBND05,04/13/2012 07:36:15.359,Makecall_01,exit,
10.173.90.34.1334320574249.1078589.OBND05,04/13/2012 07:36:15.359,EVP FatalError,enter,



-----
Again I can reload the browser and things start working again.  Will continue to run for a few weeks and then stops.  The browser only does this app and nothing else.

Any ideas?
 

 

 
 

Forgot to add - a couple days before the issue pops up I see this in an activity log as well:

10.173.90.36.1334181255013.961497.OBND05,04/11/2012 16:59:33.505,A VoiceXML Session timeout occurred at element CVP Subdialog Start: A session has timed out after 5 minutes. This is most likely caused by a start of call class or action element at the top of the callflow not completing before the voice browser's fetch timeout occurred. To resolve it ensure this class executes in a timely manner or run it in the background. Session timeouts may also occur under high load or if there are issues with a load balancer or voice browser.


Not sure if they are related or not.

Jason,
Have you looked the the memory utilization on the vxml gateways at that point ? What happens to inbound calls. Are they getting impacted in anyway when this happens. We had a similar issue and was fixed by IOS patch. The bad fetch can be caused by several reasons.
Hemal

From: Cisco Developer Community Forums [mailto:cdicuser@developer.cisco.com]
Sent: Friday, April 13, 2012 9:59 AM
To: cdicuser@developer.cisco.com
Subject: New Message from Jason Jackson in Customer Voice Portal (CVP) - CVP - All Versions: CVP Outbound App Fails at MAKECALL Element

Jason Jackson has created a new message in the forum "CVP - All Versions":

--------------------------------------------------------------
Hey guys,

We use the outbound app (.tcl polling) from our gateways to make outbound calls. The vxml browsers seem to run fine for a few weeks and then they refuse to make any more outbound calls even when we have data in the database. The only way to fix the issue is to reload the browser. I really want to figure this out. Probably an IOS bug that's not documented yet.

Anyway - in the logs on the browser we can see this:

Apr 13 07:46:32.156 CDT: //0/872DACE586A1/VXML:/vxml_block_build:
CALL_ERROR; http://removed:7000/CVP/Server
at line 49: block name _in102507 is not unique
Apr 13 07:46:32.156 CDT: //0/872DACE586A1/VXML:/vxml_parse:

Apr 13 07:46:32.156 CDT: vxml_parse: XML_Parse success err=0
Apr 13 07:46:32.156 CDT: vxml_tree_delete_for_parse_failure:mem_mgr_mempool_free: mempool=NULL
Apr 13 07:46:32.156 CDT: //0/872DACE586A1/VXML:/vxml_session_delete:

Apr 13 07:46:32.156 CDT: vxml_session_delete:mem_mgr_mempool_free: mem_refcnt(6809FAE0)=0 - mempool cleanup
Apr 13 07:46:32.156 CDT: vxml_session_delete:mem_mgr_mempool_free: mempool=NULL
Apr 13 07:46:32.156 CDT: //755615/872DACE586A1/VXML:/vxml_stop_fetchaudio_load_fail:

Apr 13 07:46:32.156 CDT: //755615/872DACE586A1/VXML:/vxml_bgload_post_done:
CALL_ERROR; http://removed:7000/CVP/Server
loaded document (http://removed:7000/CVP/Server) parse error
Apr 13 07:46:32.156 CDT: //755615/872DACE586A1/VXML:/vxml_event_proc:
<event>: event=error.badfetch status=0

In the activity logs we see this:


10.173.90.34.1334320574249.1078589.OBND05,04/13/2012 07:36:15.171,Makecall_01,custom,DNIS,(removed)10.173.90.34.1334320574249.1078589.OBND05,04/13/2012 07:36:15.175,Makecall_01,interaction,audio_group,initial_audio_group
10.173.90.34.1334320574249.1078589.OBND05,04/13/2012 07:36:15.587,Makecall_01,interaction,noinput,1
10.173.90.34.1334320574249.1078589.OBND05,04/13/2012 07:36:15.587,Makecall_01,interaction,audio_group,noinput_audio_group
10.173.90.34.1334320574249.1078589.OBND05,04/13/2012 07:36:15.359,Makecall_01,element,error,error.badfetch
10.173.90.34.1334320574249.1078589.OBND05,04/13/2012 07:36:15.359,Makecall_01,exit,
10.173.90.34.1334320574249.1078589.OBND05,04/13/2012 07:36:15.359,EVP FatalError,enter,



-----
Again I can reload the browser and things start working again. Will continue to run for a few weeks and then stops. The browser only does this app and nothing else.

Any ideas?






--
To respond to this post, please click the following link:

<http://developer.cisco.com/web/cvp/forums/-/message_boards/view_message/5451545>

or simply reply to this email.

Do you have a setting like

vxml tree memory 1000

on your gateway? This is designed so that complicated VXML apps with many elements can be loaded into the browser, but maybe there is a bug that such a setting could help with.

Regards,
Geoff

Do you have a setting like

vxml tree memory 1000

on your gateway? This is designed so that complicated VXML apps with many elements can be loaded into the browser, but maybe there is a bug that such a setting could help with.

Regards,
Geoff

 
Yes, we have that exact setting in our vxml browsers.
 
Memory utilization looks good on the boxes.
 
I went back and reviewed our syslog information and found something that popped up right before the condition started:
 
 
------
 
1 2012/04/12 14:10:12.770 EDT 10.173.93.41 26: Apr 12 13:10:11.105 CDT: %DATACORRUPTION-1-DATAINCONSISTENCY: copy error,  -PC= 0x680E1768z
2 2012/04/12 14:10:12.770 EDT 10.173.93.41 27: -Traceback= 62E9B5B0z 6266F350z 62659EB4z 6263FCB4z 626402FCz 62641C8Cz 62642D74z 6263B2E4z 62661EC8z 6266A3DCz 626BE63Cz 626C1640z 62483B38z 6249B8B0z 62409398z 62408858z
 
------
I still have the router in this condition so I'm hoping that TAC can figure this out.  I've never seen the error message above before but it doesn't appear to be very good thing when you read up on it with the bug toolkit and error lookup tool.
 
Will post more info as I can.