July 21, 2008

OEM Agent failed after the server crashed.

Production box - 9.2.0.6 database
Suse Linux 9

Agent not started up. $ORACLE_HOME/sysman/log/emagent.trc files error shows " ORA-12541: TNS:no listener" error.

The Listener port for the prod database is 1531, the above error shows that the port is configured in targets.xml is 1521.

=> Edit $ORACLE_HOME/sysman/emd/targets.xml

-> changed port value, save and exit.

=> Stop and start the agent. Failed again with Timezone error.

---------
2008-07-21 09:54:02 Thread-1098714528 ERROR pingManager: OMS does not understand the timezone region of the agent. Either start the OMS using the extended list of time zones supported by the database or pick a value of time zone from /oracle/oem_agent/agent10g/sysman/admin/nsupportedtzs.lst , update the property 'agentTZRegion' in the file /oracle/oem_agent/agent10g/sysman/config/emd.properties and restart the agent.A value which is around an offset of -04:00 from GMT should be picked.
---------

Solution
===========
1. edit the file - Agent $ORACLE_HOME/sysman/config/emd.properties
from
agentTZRegion=-04:00
to
agentTZRegion= Etc/GMT+4

oracle@nsororacle:config>env agentTZRegion emd.properties
#agentTZRegion=US/Eastern
agentTZRegion=Etc/GMT+4
oracle@nsororacle:config>

2. on the command line set -
export TZ=Etc/GMT+4

oracle@nsororacle:/oracle/oem_agent/agent10g/sysman/emd> env | grep TZ
TZ=Etc/GMT+4

NB: Steps 1 and 2 are only required to set the correct timezone

3. Log in to the repository database as SYSMAN and run (again) -
SQL> select timezone_region from mgmt_targets where target_name like '%nsororacle%';

TIMEZONE_REGION
----------------------------------------------------------------
Etc/GMT+4

4. If all three match (i.e. Etc/GMT+4) - then go to the next step

NB: Step 3 sets the timezone in the repository database - if you have done this ignore steps 1-4

5. Clean out "pending" the files in -
$AGENT ORACLE_HOME/sysman/emd

eg.
rm -r $ORACLE_HOME/sysman/emd/state/*
rm -r $ORACLE_HOME/sysman/emd/collection/*
rm -r $ORACLE_HOME/sysman/emd/upload/*
rm $ORACLE_HOME/sysman/emd/lastupld.xml
rm $ORACLE_HOME/sysman/emd/agntstmp.txt

oracle@nsororacle: emd> ls
agntstmp.txt collection lastupld.xml protocol.ini recv state targets.xml upload

oracle@nsororacle: state> rm -f *
oracle@nsororacle: state> cd ../collection
oracle@nsororacle: collection> rm -f *
oracle@nsororacle: collection> cd ../upload
oracle@nsororacle: upload> rm -f *
oracle@nsororacle: emd> rm -f lastupld.xml agntstmp.txt

6. Start the agent
$AGENT_HOME/bin/emctl start agent

oracle@nsororacle:/oracle/oem_agent/agent10g/sysman/emd> emctl start agent
Oracle Enterprise Manager 10g Release 10.2.0.1.0.
Copyright (c) 1996, 2005 Oracle Corporation. All rights reserved.
Starting agent ......... started.

7. Issue an agent clearstate from the agent home
$AGENT_HOME/bin/emctl clearstate agent

oracle@nsororacle:/oracle/oem_agent/agent10g/sysman/emd>emctl clearstate agent Oracle Enterprise Manager 10g Release 10.2.0.1.0.
Copyright (c) 1996, 2005 Oracle Corporation. All rights reserved.
EMD clearstate completed successfully

8. Force an upload to the OMS
$AGENT_HOME/bin/emctl upload agent

oracle@nsororacle:/oracle/oem_agent/agent10g/sysman/emd> emctl upload agent
Oracle Enterprise Manager 10g Release 10.2.0.1.0.
Copyright (c) 1996, 2005 Oracle Corporation. All rights reserved.
---------------------------------------------------------------
EMD upload completed successfully.

The agent should stop crashing now.

oracle@nsororacle:/oracle/oem_agent/agent10g/sysman/emd> emctl status agent
Oracle Enterprise Manager 10g Release 10.2.0.1.0.
Copyright (c) 1996, 2005 Oracle Corporation. All rights reserved.
---------------------------------------------------------------
Agent Version : 10.2.0.1.0
OMS Version : 10.2.0.1.0
Protocol Version : 10.2.0.0.0
Agent Home : /oracle/oem_agent/agent10g
Agent binaries : /oracle/oem_agent/agent10g
Agent Process ID : 24811
Parent Process ID : 24796
Agent URL : https://nsororacle.corp.nsoro.com:3872/emd/main/
Repository URL : https://nsororacle2.corp.nsoro.com:1159/em/upload
Started at : 2008-07-21 12:28:55
Started by user : oracle
Last Reload : 2008-07-21 12:28:55
Last successful upload : 2008-07-21 12:30:44
Total Megabytes of XML files uploaded so far : 6.10
Number of XML files pending upload : 0
Size of XML files pending upload(MB) : 0.00
Available disk space on upload filesystem : 39.77%
Last successful heartbeat to OMS : 2008-07-21 12:30:03
---------------------------------------------------------------
Agent is Running and Ready
oracle@nsororacle:/oracle/oem_agent/agent10g/sysman/emd>


=============================
Reference: Note:452481.1 - Problem: Agent Stops: The Agent Starts but crashes shortly afterwards - timezone change
=============================

No comments: