Oracle Enterprise Manager 13.3 problems discovering Exadata targets

If you are on the latest Exadata plugin for EM 13.3 (well, even the 13.2 version of the plugin). Be aware that there’s a bug where if the Exadata nodes’ names are longer than 7 characters, you cannot discover them.

EM just throws a weird error with below legend:

SEVERE: makeTargetsXml for each rack#: 1 e=java.lang.StringIndexOutOfBoundsException: String index out of range: -1 

The solution is to apply the latest (March 2019) Bundle Patch to the agents running on the Exadata’s compute nodes.

More info on below MOS note:  

Exadata Discovery Fails With “No New Component Is Discovered. Click Cancel” (Doc ID 2477624.1)  

Another good excuse to keep your EM system up to date.  

Happy patching.  


OMS responded illegally [ERROR- Failed to Update Target Type Metadata] in 13c R2

I want to discuss about this OMS error coming from the agent’s heartbeat status. The agent shows with an unreachable status but is actually up and running.
$: emctl status agent
Oracle Enterprise Manager Cloud Control 13c Release 2
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Agent Version          :
OMS Version            : (unknown)
Last Reload            : (none)
Last successful upload                       : (none)
Last attempted upload                        : (none)
Total Megabytes of XML files uploaded so far : 0
Number of XML files pending upload           : 2,210
Size of XML files pending upload(MB)         : 2.89
Available disk space on upload filesystem    : 56.39%
Collection Status                            : Collections enabled
Heartbeat Status       : OMS responded illegally [ERROR- Failed to Update Target Type Metadata]
Last attempted heartbeat to OMS              : 2017-01-24 21:09:21
Last successful heartbeat to OMS             : (none)
Next scheduled heartbeat to OMS              : 2017-01-24 21:09:51
Agent is Running and Ready
In this case the agent is not able to upload the metadata because is different in the OMS.
This is mainly due to a patch in the agent side that is missing in the OMS. By mistake I applied the latest bundle patch to the agent before applying it to the OMS.
Just another proof that any patches have to be installed to the OMS first then the agents.


Upgrade OPatch for OEM 13c R2 Agents

Opatch utility has to be updated often as Oracle delivers regular updates to it. I was used to version 12 and it only required an overwrite of the OPatch directory. Well, not anymore with version 13c R2.
I will walk you through an OPatch upgrade from 13.8 to 13.9 version.
First export your agent ORACLE_HOME:
$: export ORACLE_HOME=/u01/oracle/oemagent/agent_13.
Verify the current OPatch version:
$: $ORACLE_HOME/OPatch/opatch version
OPatch Version:
OPatch succeeded.
Download and unzip the 13.9 version in a temporary directory:
$: unzip
   creating: 6880880/
  inflating: 6880880/README.txt
  inflating: 6880880/opatch_generic.jar
  inflating: 6880880/version.txt
As you can see, now we have some zip files and a java file. Execute java to upgrade OPatch as follows:
$: $ORACLE_HOME/jdk/bin/java -jar opatch_generic.jar -silent oracle_home=$ORACLE_HOME
Launcher log file is /tmp/OraInstall2017-01-19_08-47-21PM/launcher2017-01-19_08-47-21PM.log.
Extracting the installer . . . . Done
Checking if CPU speed is above 300 MHz.   Actual 3000.000 MHz    Passed
Checking swap space: must be greater than 512 MB.   Actual 16383 MB    Passed
Checking if this platform requires a 64-bit JVM.   Actual 64    Passed (64-bit not required)
Checking temp space: must be greater than 300 MB.   Actual 18842 MB    Passed
Preparing to launch the Oracle Universal Installer from /tmp/OraInstall2017-01-19_08-47-21PM
Installation Summary
Disk Space : Required 6 MB, Available 5,229 MB
Feature Sets to Install:
        Next Generation Install Core
        OPatch Auto OPlan
Session log file is /tmp/OraInstall2017-01-19_08-47-21PM/install2017-01-19_08-47-21PM.log
Loading products list. Please wait.
Loading products. Please wait.
Updating Libraries
Starting Installations
Install pending
Installation in progress
 Component : oracle.swd.opatch
Copying files for ‘oracle.swd.opatch ‘
 Component : oracle.glcm.osys.core
Copying files for ‘oracle.glcm.osys.core ‘
 Component : oracle.glcm.oplan.core
Copying files for ‘oracle.glcm.oplan.core ‘
 Component : oracle.glcm.opatch.common.api
Copying files for ‘oracle.glcm.opatch.common.api ‘
 Component : oracle.glcm.opatchauto.core
Copying files for ‘oracle.glcm.opatchauto.core ‘
Install successful
Post feature install pending
Post Feature installing
 Feature Set : apache_commons_cli_lib
 Feature Set : oracle.glcm.opatchauto.core.actions.classpath
Post Feature installing ‘apache_commons_cli_lib’
 Feature Set : oracle.glcm.opatchauto.core.binary.classpath
 Feature Set : oracle.glcm.osys.core.classpath
Post Feature installing ‘oracle.glcm.opatchauto.core.binary.classpath’
Post Feature installing ‘oracle.glcm.osys.core.classpath’
 Feature Set : apache_commons_compress_lib
 Feature Set : oracle.glcm.opatchauto.core.wallet.classpath
Post Feature installing ‘oracle.glcm.opatchauto.core.actions.classpath’
Post Feature installing ‘apache_commons_compress_lib’
 Feature Set : oracle.glcm.opatchauto.core.classpath
Post Feature installing ‘oracle.glcm.opatchauto.core.wallet.classpath’
Post Feature installing ‘oracle.glcm.opatchauto.core.classpath’
 Feature Set : oracle.glcm.opatch.common.api.classpath
 Feature Set : oracle.glcm.oplan.core.classpath
Post Feature installing ‘oracle.glcm.opatch.common.api.classpath’
Post Feature installing ‘oracle.glcm.oplan.core.classpath’
Post feature install complete
String substitutions pending
String substituting
 Component : oracle.swd.opatch
String substituting ‘oracle.swd.opatch ‘
 Component : oracle.glcm.osys.core
String substituting ‘oracle.glcm.osys.core ‘
 Component : oracle.glcm.oplan.core
String substituting ‘oracle.glcm.oplan.core ‘
 Component : oracle.glcm.opatch.common.api
String substituting ‘oracle.glcm.opatch.common.api ‘
 Component : oracle.glcm.opatchauto.core
String substituting ‘oracle.glcm.opatchauto.core ‘
String substitutions complete
Link pending
Linking in progress
 Component : oracle.swd.opatch
Linking ‘oracle.swd.opatch ‘
 Component : oracle.glcm.osys.core
Linking ‘oracle.glcm.osys.core ‘
 Component : oracle.glcm.oplan.core
Linking ‘oracle.glcm.oplan.core ‘
 Component : oracle.glcm.opatch.common.api
Linking ‘oracle.glcm.opatch.common.api ‘
 Component : oracle.glcm.opatchauto.core
Linking ‘oracle.glcm.opatchauto.core ‘
Linking in progress
Link successful
Setup pending
Setup in progress
 Component : oracle.swd.opatch
Setting up ‘oracle.swd.opatch ‘
 Component : oracle.glcm.osys.core
Setting up ‘oracle.glcm.osys.core ‘
 Component : oracle.glcm.oplan.core
Setting up ‘oracle.glcm.oplan.core ‘
 Component : oracle.glcm.opatch.common.api
Setting up ‘oracle.glcm.opatch.common.api ‘
 Component : oracle.glcm.opatchauto.core
Setting up ‘oracle.glcm.opatchauto.core ‘
Setup successful
Save inventory pending
Saving inventory
Saving inventory complete
Configuration complete
Logs successfully copied to /u01/app/oraInventory/logs.
Verify the new OPatch version

$: $ORACLE_HOME/OPatch/opatch version
OPatch Version:
OPatch succeeded.