[ Issue ] Concurrent Manager startup errors out

Hello All,
Few days back , we encountered an error while starting Concurrent Managers ;
Here is the log file [segment]
Error :
=========================================================
Check that your system has enough resources to start a concurrent manager process. Contact your system administ : 18-AUG-2009 13:44:33
Starting PASMGR Concurrent Manager : 18-AUG-2009 13:44:33
CONC-SM TNS FAIL
Routine AFPEIM encountered an error while starting concurrent manager PASMGR with library /m03/DEV/apps/apps_st/appl/pa/12.0.0/bin/PALIBR.
Check that your system has enough resources to start a concurrent manager process. Contact your system administrato : 18-AUG-2009 13:44:33
Process monitor session ended : 18-AUG-2009 13:44:33
Process monitor session started : 18-AUG-2009 13:46:33
Could not contact Service Manager FNDSM_APPSDBA1_DEV. The TNS alias could not be located, the listener process on APPSDBA1 could not be contacted, or the listener failed to spawn the Service Manager process.
Could not contact Service Manager FNDSM_APPSDBA_DEV. The TNS alias could not be located, the listener process on APPSDBA could not be contacted, or the listener failed to spawn the Service Manager process.
Starting RCVOLTM Concurrent Manager : 18-AUG-2009 13:46:33CONC-SM TNS FAILRoutine AFPEIM encountered an error while starting concurrent manager RCVOLTM with library /u03/DEV/apps/apps_st/appl/po/12.0.0/bin/RCVOLTM.
Check that your system has enough resources to start a concurrent manager process.
Contact your system administra : 18-AUG-2009 13:46:33
Starting PODAMGR Concurrent Manager : 18-AUG-2009 13:46:33CONC-SM TNS FAIL
=========================================================

Cause :
Some of the libraries of FNDSM/FNDLIBR must be missing or corrupted.

Solution :
1] Shutdown all the services.
2] Log in as applmgrcd to $FND_TOP/patch/115/sql
Run the script: afdcm037.sql
3] Relink FNDSM and FNDLIBR executables as mentioned below:
$ adrelink.sh force=y link_debug=y "fnd FNDLIBR"
$ adrelink.sh force=y link_debug=y "fnd FNDSM"
4] Run cmclean.sql
5] Start up the Services and retest.

Refer :
Metalink note-id : 460578.1 , 555081.1

[Issue] Concurrent Managers Not Starting With GSM Enabled

We encountered an error on our Production Instance[12.0.4], though all the services were up and running .. we could see only single process of FNDLIBR running.
When we checked the Concurrent:GSM Enabled profile option value ; it was set to "Y".

Error ::
All requests show a status as 'Inactive No Manager' .

Cause ::
The following is one possible cause for failure of concurrent managers to start:
Inability to ping the concurrent manager node after logging in as the OS user who starts the concurrent managers.
'ping ' returns the following error:
ping: socket Permission denied
The application does a ping in order to check if the node is active.
If the ping fails, it adds the node as inactive. This prevents the concurrent managers from starting successfully.

Solution ::
The command 'ls -l ping' should return the following OS permissions in the directory where the ping command resides:
-r-sr-xr-x 1 root bin
If the permissions are different, issue the following command as root user:
'chmod 4555 ping'.
This will ensure that the OS user who starts the concurrent managers is able to ping successfully.

Refer :: Metalink note-id :311935.1