Upgrade Your DB to 19c As A PDB using Enterprise Manager 13c

With Oracle Database 19c being the long term release, makes sense to start planning the upgrade of the DBs running on previous releases. Even better, what about converting Non-CDB databases into PDBs at the same time?

Using Oracle Enterprise Manager (EM) 13c, we can kill two birds with one stone. That is upgrade our DBs to 19c and migrate them into a PDB at the same time.

How this works? First of all, this functionality is part of the Database Lifecycle Management Pack. You can find the documentation for this feature along with all the pre-reqs below:



Let me guide you through this process.

In this environment I have a single instance database “orders”. As you can see on the screenshot below, is currently running version 18.10.

Orders DB Running 18.10 And Db19c Running 19.3 Version

Let’s now navigate to Enterprise -> Provisioning and Patching -> Database Provisioning

Database Provisioning Menu

Locate the Provision Pluggable Databases link under Related Links section

Provision Pluggable Databases Link

This will open the Provision Pluggable Database dashboard.

Provision Pluggable Databases Dashboard

Select the Container Database which is out target. Click on Migrate Existing Databases and then click the Launch button.

Select the desired migration method. You can choose between Data Pump Full Transportable Export and Import or Plug as a PDB. There are cons and pros on each method. Review the documentation previously shared so you can choose the best option for your migration.

For the purpose of this example we are going to choose Data Pump Full Transportable Export and Import.

Available Migration Methods

Now add the source DBs to be migrated.

Add DBs To Be Migrated
Select The Orders DB

Add or select the required Named Credentials and provide the path for the Export Directory.

Source Information

Provide the required Destination Information and click Next.

Destination Information

EM will execute compatibility and validation checks. If there are no errors click Close to proceed.

Validation Completed Successfully

Type a name for the EM job and select the desired schedule for it to run.

Schedule Information

Review all the information provided and click Submit.

Review Window

Wait for the Confirmation pop-up window and click View Job.

Confirmation Window

Monitor the progress of the deployment procedure until it completes.

Migration Deployment Procedure

Confirm that the db19c container DB now has the ORDERS PDB.

Orders PDB – db19c CDB

Click on the ORDERS PDB and verify the version information.

ORDERS PDB Dashboard

We now have migrated our orders DB into the db19c container running 19.3 version.

I also recommend to take a look at MOS Doc ID 2694986.1 before attempting the migration.

Hope this helps to ease the migration process for your DBs.

Thanks,
Alfredo

Upgrade Oracle Enterprise Manager to 13.5

Oracle Enterprise Manager 13.5 (EM) was just recently announced. The installation files along with the documentation are now available in the Oracle portal. The upgrade process is very similar to the one followed for 13.4.

You can find the process I followed to upgrade EM to 13.4 here:



Let’s now get into the 13.5 upgrade process.

The same way as EM 13.4, this is an out of place upgrade. What is the significance of this? Well, you have to install a new Middleware home. Then the wizard will take care of all the required steps to upgrade EM using this new home.

My current environment details:

  • Oracle Enterprise Manager 13.4 RU 9 (Single OMS)
  • Oracle Database 19c RU 9 (OMR)
  • Oracle Linux 7 64-bit

Take a look at the

Oracle EM documentation and make sure you comply with all the pre-requisites before attempting the upgrade.



It is recommended that you apply the latest RU to the Oracle Database hosting the Oracle Management Repository (OMR). Review and make sure you meet all the pre-requisites detailed in the upgrade guide.

Download the Oracle EM 13.5 software and stage it on your OMS server.



$ chmod +x em13500_linux64.bin
$ ./em13500_linux64.bin

Select an installation type. In this case will be an End-to-End upgrade. As you can see, the installer has been enhanced compared with the previous release.

I’m going to un-select the security updates box and click Next.

The wizard will run through a couple of checks and provide recommendations.

Review the output of each check and perform any required fix.

Type the location for the new Middleware home.

Enter the Database connection details for the OMR.

Fix the required Database checks.

You can click the Auto Fix button. Auto Fix won’t fix things you have to manually setup. In my case I had to execute the following:

  • Copy the emkey to repos
$ ./emctl config emkey -copy_to_repos
Oracle Enterprise Manager Cloud Control 13c Release 4
Copyright (c) 1996, 2020 Oracle Corporation.  All rights reserved.
Enter Enterprise Manager Root (SYSMAN) Password :
The EMKey has been copied to the Management Repository. This operation will cause the EMKey to become unsecure.
After the required operation has been completed, secure the EMKey by running "emctl config emkey -remove_from_repos".
  • Shutdown the EM agent
$ cd $AGENT_HOME/bin
$ ./emctl stop agent
  • Shutdown OMS
$ cd $ORACLE_HOME/bin
$ ./emctl stop oms -all
Oracle Enterprise Manager Cloud Control 13c Release 4
Copyright (c) 1996, 2020 Oracle Corporation.  All rights reserved.
Stopping Oracle Management Server...
WebTier Successfully Stopped
Oracle Management Server Successfully Stopped
Oracle Management Server is Down
JVMD Engine is Down
Stopping BI Publisher Server...
BI Publisher Server Successfully Stopped
AdminServer Successfully Stopped
BI Publisher Server is Down

I ignored the MD5 check.

As previously stated. You have to verify that the DB hosting the OMR has all the required patched already applied.

Verify the plug-ins that are going to be upgraded to 13.5

Select additional plug-ins you want to install as part of this upgrade to 13.5

Type the WebLogic password and the location of the WebLogic base instance

Review all the upgrade inputs and click the Upgrade button

The upgrade wizard will perform all the required steps

Execute root scripts

$ sudo /u01/app/em135/middleware/allroot.sh

Starting to execute allroot.sh .........

Starting to execute /u01/app/em135/middleware/root.sh ......
Check /u01/app/em135/middleware/install/root_emcc.marketplace.com_2021-05-10_23-16-24.log for the output of root script
Finished execution of  /u01/app/em135/middleware/root.sh ......

Oracle EM has been upgraded to 13.5

Now you should start you EM agent and start upgrading all of them.

All new features on 13.5 are described below.



As you can see, the process is very straight forward.

Happy upgrade!!!

Thanks,
Alfredo

Applying Oracle Enterprise Manager 13.4 RU 8

Oracle Enterprise Manager (EM) Release Update (RU) 8 for 13.4 version was released at the end of last month. Follow the link below to find out what’s included in this RU.



In this post, I want to show you how to apply this RU to your Oracle Enterprise Manager environment.

Let me start by describing the environment that I’m using for this. Oracle Enterprise Manager 13.4 can be deployed both on-premises or in the Oracle Cloud (OCI). Oracle Enterprise Manager 13.4 is available in the OCI’s Marketplace. You have the ability to choose from both single-instance and multi-node EM deployment. This makes the process really straightforward and you can have an environment up and running with couple of clicks.



For this example I’m going to be using the on-premises option. This means that Oracle EM 13.4 is also available for download as a VirtualBox image. This image comes as a single-instance EM deployment. Is worth to mention that this image is not recommended for anything but for testing.



I assigned 16 Gb of RAM and 2 virtual CPUs to this VirtualBox VM. Follow the instructions in the README file and start your environment. I also downloaded patch 32071974 from MOS Doc ID 2647078.1

For this RU, there are specific instructions about OPatch and OMSPatcher that we need to take attention to. The first one is that we need to upgrade OPatch and immediately after that apply the RU. If you are not planning to apply RU8 right away the advice is not to upgrade OPatch as the MOS note mentions that it may disrupt the environment.

Go ahead and download patch 28186730. Unzip the patch file into an staging area and execute the OPatch upgrade tool. Be sure that you have a valid backup of both, the Oracle Home and the EM configuration (export config).

$ $ORACLE_HOME/oracle_common/jdk/bin/java -jar /home/oracle/patches/6880880/opatch_generic.jar -silent oracle_home=$ORACLE_HOME
Launcher log file is /tmp/OraInstall2020-12-07_04-35-38PM/launcher2020-12-07_04-35-38PM.log.
Extracting the installer . . . . Done
Checking if CPU speed is above 300 MHz.   Actual 2245.782 MHz    Passed
Checking swap space: must be greater than 512 MB.   Actual 17151 MB    Passed
Checking if this platform requires a 64-bit JVM.   Actual 64    Passed (64-bit not required)
Checking temp space: must be greater than 300 MB.   Actual 25737 MB    Passed
Preparing to launch the Oracle Universal Installer from /tmp/OraInstall2020-12-07_04-35-38PM
Installation Summary


Disk Space : Required 34 MB, Available 37,853 MB
Feature Sets to Install:
        Next Generation Install Core 13.9.4.0.1
        OPatch 13.9.4.2.5
        OPatch Auto OPlan 13.9.4.2.5
Session log file is /tmp/OraInstall2020-12-07_04-35-38PM/install2020-12-07_04-35-38PM.log

Loading products list. Please wait.
 1%
 40%

Loading products. Please wait.
 42%
 43%
 45%
 46%
 48%
 49%
 50%
 51%
 52%
 54%
 55%
 57%
 58%
 60%
 61%
 62%
 64%
 65%
 67%
 68%
 70%
 71%
 72%
 74%
 75%
 77%
 78%
 80%
 81%
 82%
 84%
 85%
 87%
 88%
 90%
 91%
 92%
 94%
 95%
 97%
 98%
 99%

Updating Libraries



Starting Installations
 1%
 2%
 3%
 4%
 5%
 6%
 7%
 8%
 9%
 10%
 11%
 12%
 13%
 14%
 15%
 16%
 17%
 18%
 19%
 20%
 21%
 22%
 23%
 24%
 25%
 26%
 27%
 28%
 29%
 30%
 31%
 32%
 33%
 34%
 35%
 36%
 37%
 38%
 39%
 40%
 41%
 42%
 43%
 44%
 45%
 46%
 47%
 48%
 49%
 50%
 51%
 52%
 53%
 54%
 55%
 56%
 57%
 58%
 59%
 60%
 61%
 62%
 63%
 64%
 65%
 66%
 67%
 68%
 69%
 70%
 71%
 72%
 73%
 74%
 75%
 76%
 77%
 78%
 79%
 80%
 81%
 82%
 83%
 84%
 85%
 86%
 87%
 88%
 89%
 90%
 91%
 92%
 93%
 94%
 95%

Install pending
...
Saving the inventory oracle.glcm.opatch.common.api.classpath

The install operation completed successfully.

Logs successfully copied to /u01/app/oraInventory/logs.

Now is time to upgrade OMSPatcher to the latest version. Follow the instructions from MOS Doc ID 2646080.1.

$ omspatcher version
OMSPatcher Version: 13.9.4.4.0
OPlan Version: 12.2.0.1.16
OsysModel build: Tue Nov 03 18:16:07 PST 2020

OMSPatcher succeeded.

After that, you will be ready to start patching.

Let’s now go to the directory where we are staging our patch and execute the OMSPatcher analyze command. For this RU, I downloaded patch 32071974 and extracted into a staging directory in the server.

$ omspatcher apply -analyze
OMSPatcher Automation Tool
Copyright (c) 2017, Oracle Corporation.  All rights reserved.


OMSPatcher version : 13.9.4.4.0
OUI version        : 13.9.4.0.0
Running from       : /u01/app/oracle/product/em134/middleware
Log file location  : /u01/app/oracle/product/em134/middleware/cfgtoollogs/omspatcher/opatch2020-12-07_16-49-21PM_1.log

OMSPatcher log file: /u01/app/oracle/product/em134/middleware/cfgtoollogs/omspatcher/32071974/omspatcher_2020-12-07_16-49-26PM_analyze.log

...

OMSPatcher Session completed with warnings.
Log file location: /u01/app/oracle/product/em134/middleware/cfgtoollogs/omspatcher/32071974/omspatcher_2020-12-07_16-49-26PM_analyze.log

OMSPatcher completed with warnings.

Those warnings are expected as I don’t have some of those plug-ins deployed in this environment.

After you validate that there are no conflicts. Proceed to shutdown the Oracle Management Service (OMS).

$ emctl stop oms
Oracle Enterprise Manager Cloud Control 13c Release 4
Copyright (c) 1996, 2020 Oracle Corporation.  All rights reserved.
Stopping Oracle Management Server...
Oracle Management Server Successfully Stopped
Oracle Management Server is Down
JVMD Engine is Down

And now let’s execute the OMSPatcher command to apply the RU patch.

$ omspatcher apply
OMSPatcher Automation Tool
Copyright (c) 2017, Oracle Corporation.  All rights reserved.


OMSPatcher version : 13.9.4.4.0
OUI version        : 13.9.4.0.0
Running from       : /u01/app/oracle/product/em134/middleware
Log file location  : /u01/app/oracle/product/em134/middleware/cfgtoollogs/omspatcher/opatch2020-12-07_16-52-59PM_1.log

OMSPatcher log file: /u01/app/oracle/product/em134/middleware/cfgtoollogs/omspatcher/32071974/omspatcher_2020-12-07_16-53-05PM_deploy.log

...

--------------------------------------------------------------------------------
The following warnings have occurred during OPatch execution:
1)  Sub-patch "31812958" skipped as it is already been applied to the OMS system.
2)  Could not apply the patch "31802583" because the "oracle.sysman.vt.oms.plugin with version 13.4.1.0.0" core component of the OMS or the plug-in for which the patch is intended is either not deployed or deployed with another version in your Enterprise Manager system.
3)  Could not apply the patch "31894733" because the "oracle.sysman.am.oms.plugin with version 13.4.1.0.0" core component of the OMS or the plug-in for which the patch is intended is either not deployed or deployed with another version in your Enterprise Manager system.
4)  Could not apply the patch "30851102" because the "oracle.sysman.vi.oms.plugin with version 13.4.1.0.0" core component of the OMS or the plug-in for which the patch is intended is either not deployed or deployed with another version in your Enterprise Manager system.
5)  Could not apply the patch "31894749" because the "oracle.sysman.bda.oms.plugin with version 13.4.1.0.0" core component of the OMS or the plug-in for which the patch is intended is either not deployed or deployed with another version in your Enterprise Manager system.
6)  Could not apply the patch "31894832" because the "oracle.sysman.ssa.oms.plugin with version 13.4.1.0.0" core component of the OMS or the plug-in for which the patch is intended is either not deployed or deployed with another version in your Enterprise Manager system.
7)  Could not apply the patch "31894720" because the "oracle.sysman.emfa.oms.plugin with version 13.4.1.0.0" core component of the OMS or the plug-in for which the patch is intended is either not deployed or deployed with another version in your Enterprise Manager system.
8)  Could not apply the patch "31889317" because the "oracle.sysman.csm.oms.plugin with version 13.4.1.0.0" core component of the OMS or the plug-in for which the patch is intended is either not deployed or deployed with another version in your Enterprise Manager system.
9)  Could not apply the patch "31802539" because the "oracle.sysman.smf.oms.plugin with version 13.4.1.0.0" core component of the OMS or the plug-in for which the patch is intended is either not deployed or deployed with another version in your Enterprise Manager system.
--------------------------------------------------------------------------------
OMSPatcher Session completed with warnings.
Log file location: /u01/app/oracle/product/em134/middleware/cfgtoollogs/omspatcher/32071974/omspatcher_2020-12-07_16-53-05PM_deploy.log

OMSPatcher completed with warnings.

Verify the output of OMSPatcher. Let’s start the OMS.

$ emctl start oms
Oracle Enterprise Manager Cloud Control 13c Release 4
Copyright (c) 1996, 2020 Oracle Corporation.  All rights reserved.
Starting Oracle Management Server...
WebTier Successfully Started
Oracle Management Server Successfully Started
Oracle Management Server is Up
JVMD Engine is Up
Starting BI Publisher Server ...
BI Publisher Server Already Started
BI Publisher Server is Up

Login now to the OEM console and verify that EM is showing the latest RU.

Happy patching,
Alfredo

Adobe Flash End-Of-Life and Oracle Enterprise Manager

Adobe announced the End-Of-Life (EOL) for the Flash technology and this is happening December 31, 2020. As a result, most of the popular Web browsers also announced that Flash will be blocked the same day.

I’ve been blogging about this for more than a year now, explaining how this impacts Oracle Enterprise Manager’s (EM) database performance pages.



If you are running EM on versions 13.3.1.0 and below you should upgrade now before browsers can’t display the database performance graphs.

After you upgrade to 13.3.2.0 (13.3 PG) or higher (13.4 is the recommended version), you have to get familiar with the fact that the Top Activity page has been deprecated. I also blogged about this last year.



If you are considering upgrading to Oracle EM 13.4, also consider applying the latest Release Update (RU), at the time of this blog RU8 is the latest one. Take a look at MOS Doc ID 2647078.1.

I recommend you take a look at the November Advisor Webcast for Oracle Enterprise Manager. Kurt from the Product Manager team did a great job explaining this change.

Advisor Webcast Recording: Enterprise Manager (EM) – New OEM JET Pages and Flash End Of Life held on November 19, 2020 (Doc ID 2729947.1)

Thanks,
Alfredo

Oracle Database Consolidation Workbench

During database migrations, Oracle Database Administrators (DBA) face a series of challenges that could make the migration success unpredictable. Oracle Real Application Testing pack combined with Oracle Enterprise Manager 13c offers a tool that simplifies and predicts the database migration and consolidation process.

Oracle Database Consolidation Workbench enables DBA’s to create consolidation projects and scenarios where they can input different variables and run different analysis before even attempting the migration. This helps predicts the migration and consolidation process, but it doesn’t end there. Oracle Enterprise Manager 13c also implements the migration and validates the migration using SQL Performance Analyzer (SPA).

Below is a 10 minute overview video of the Oracle Database Consolidation Workbench.

Thanks,
Alfredo

Oracle Enterprise Manager 13c – Connection to MOS not working

Oracle Enterprise Manager (EM) 13c has the ability to connect to My Oracle Support (MOS) in order to download patch recommendations, information about new plugins and versions. This is pretty useful in terms of keeping your EM up to date.

There was an issue early this year that prevented the connection from EM to the MOS portal. This has been addressed and a patch is needed in your EM in order to restore this functionality.

Take a look at patch 31233849 and MOS note 2697356.1. This patch is already included on the EM 13c R6 update. My recommendation is to apply the latest RU to your EM 13.4 installation.

Happy patching!!!

Thanks,
Alfredo

Oracle Enterprise Manager Webcast Series

This year we started a series of deep dive Webcasts related to Oracle Enterprise Manager 13c including all the monitoring, tuning and manageability tools that the Database Management Packs offer.

I want to condense a list of resources for you so you can take advantage of these Webcasts either live or watching the recorded session.

But there’s still more to come. Next month we are going to be hosting another set of Webcast sessions for you. Don’t forget to register!

Thanks,
Alfredo

Oracle Enterprise Manager App for Grafana

Oracle Enterprise Manager 13c provides out-of-box dashboard functionality, so you can see the status of all your targets on a single screen. You also have information about target’s fatal, critical, warning and escalated incidents along with detailed information about them.

But what about additional data that EM is already collecting? What if you create Metric Extensions and you want to display them in the dashboard?

Oracle Enterprise Manager App for Grafana was just released last month. This Grafana plug-in allows you directly connect from Grafana to Oracle Enterprise Manager and make use of the already collected data in order to create custom dashboards.



In this post I want to guide you through the process of installing a test Grafana server and connect that to Oracle Enterprise Manager.

First things first, you need to make sure you install Grafana 6.6.2 or higher and you have Oracle Enterprise Manager 13.4 RU 3 or higher. Let’s now jump into the installation process.

I have a small VM running OL7. I assinged 2 CPUs and 8 GB of RAM. The next step is to install Grafana in the VM.

$ wget https://dl.grafana.com/oss/release/grafana-7.0.5-1.x86                                                                                                                                                                                                                                             _64.rpm
--2020-06-30 19:36:01--  https://dl.grafana.com/oss/release/grafana-7.0.5-1.x86_                                                                                                                                                                                                                                             64.rpm
Resolving dl.grafana.com (dl.grafana.com)... 151.101.138.217, 2a04:4e42:46::729
Connecting to dl.grafana.com (dl.grafana.com)|151.101.138.217|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 49118311 (47M) [application/x-redhat-package-manager]
Saving to: ‘grafana-7.0.5-1.x86_64.rpm’

100%[======================================>] 49,118,311   232MB/s   in 0.2s

2020-06-30 19:36:02 (232 MB/s) - ‘grafana-7.0.5-1.x86_64.rpm’ saved [49118311/49                                                                                                                                                                                                                                             118311]
$ sudo yum install grafana-7.0.5-1.x86_64.rpm
Loaded plugins: langpacks, ulninfo
Examining grafana-7.0.5-1.x86_64.rpm: grafana-7.0.5-1.x86_64
Marking grafana-7.0.5-1.x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package grafana.x86_64 0:7.0.5-1 will be installed
--> Processing Dependency: fontconfig for package: grafana-7.0.5-1.x86_64
--> Processing Dependency: urw-fonts for package: grafana-7.0.5-1.x86_64
...
--> Running transaction check
---> Package libXau.x86_64 0:1.0.8-2.1.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package                              Arch   Version         Repository    Size
================================================================================
Installing:
 grafana                              x86_64 7.0.5-1         /grafana-7.0.5-1.x8                                                                                                                                                                                                                                             6_64
                                                                          143 M
Installing for dependencies:
 ...
Installed:
  grafana.x86_64 0:7.0.5-1

Complete!

Now is time to start Grafana server in the VM.

$ sudo systemctl daemon-reload
$ sudo systemctl start grafana-server
$ sudo systemctl status grafana-server
● grafana-server.service - Grafana instance
   Loaded: loaded (/usr/lib/systemd/system/grafana-server.service; disabled; vendor preset: disabled)
   Active: active (running) since Tue 2020-06-30 19:40:40 GMT; 1s ago
 ...
lvl=info msg="External plugins directory created" logger=plugins directory=/var/lib/grafana/plugins
Jun 30 19:40:40 grafana-s02 systemd[1]: Started Grafana instance.
Jun 30 19:40:40 grafana-s02 grafana-server[11614]: t=2020-06-30T19:40:40+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=[::]:3000 protocol=http subUrl= socket=

Let’s now login to our Grafana server installation. By default the port assigned is 3000, so the URL will be similar to http://<grafana_server>:3000

Grafana Welcome Page

The default credentials are admin/admin. Grafana will ask you to change the admin password right after the first login.

The next step is to download and install the Oracle Enterprise Manager App for Grafana. You can find the download link and documentation below.



There are 2 methods for installing the EM App for Grafana:

  • Using CLI (grafana-cli)
  • Without CLI

For my test, I’m going to install the EM App for Grafana without using the CLI.

$ sudo mv oracle-emcc-app-1.0.2.zip /var/lib/grafana/plugins/
$ sudo unzip /var/lib/grafana/plugins/oracle-emcc-app-1.0.2.zip
Archive:  oracle-emcc-app-1.0.2.zip
   creating: oracle-emcc-app-1.0.2/
   creating: oracle-emcc-app-1.0.2/datasource/
  inflating: oracle-emcc-app-1.0.2/datasource/query_ctrl.js
  inflating: oracle-emcc-app-1.0.2/datasource/Response.js
  inflating: oracle-emcc-app-1.0.2/datasource/EMCCConstants.js
  inflating: oracle-emcc-app-1.0.2/datasource/plugin.json
  inflating: oracle-emcc-app-1.0.2/datasource/QueryResource.js
   creating: oracle-emcc-app-1.0.2/datasource/img/
  inflating: oracle-emcc-app-1.0.2/datasource/img/em_logo.png
  inflating: oracle-emcc-app-1.0.2/datasource/module.js
  inflating: oracle-emcc-app-1.0.2/datasource/BaseQueryResource.js
  inflating: oracle-emcc-app-1.0.2/datasource/MetricQueryResource.js
  inflating: oracle-emcc-app-1.0.2/datasource/Utils.js
  inflating: oracle-emcc-app-1.0.2/datasource/config_ctrl.js
  inflating: oracle-emcc-app-1.0.2/datasource/datasource.js
   creating: oracle-emcc-app-1.0.2/datasource/partials/
  inflating: oracle-emcc-app-1.0.2/datasource/partials/query.editor.html
  inflating: oracle-emcc-app-1.0.2/datasource/partials/config.html
  inflating: oracle-emcc-app-1.0.2/datasource/partials/annotation.editor.html
  inflating: oracle-emcc-app-1.0.2/plugin.json
  inflating: oracle-emcc-app-1.0.2/README.md
   creating: oracle-emcc-app-1.0.2/img/
  inflating: oracle-emcc-app-1.0.2/img/em_logo.png
   creating: oracle-emcc-app-1.0.2/components/
  inflating: oracle-emcc-app-1.0.2/components/config.js
   creating: oracle-emcc-app-1.0.2/components/partials/
  inflating: oracle-emcc-app-1.0.2/components/partials/config.html
  inflating: oracle-emcc-app-1.0.2/module.js
   creating: oracle-emcc-app-1.0.2/dashboards/
  inflating: oracle-emcc-app-1.0.2/dashboards/template_single_em_single_target_type_single_target_name_single_named_credential.json
  inflating: oracle-emcc-app-1.0.2/dashboards/template_single_em_single_target_type_single_target_name.json
  inflating: oracle-emcc-app-1.0.2/dashboards/template_single_em_single_target_type.json
  inflating: oracle-emcc-app-1.0.2/dashboards/template_multi_em.json
  inflating: oracle-emcc-app-1.0.2/dashboards/database_performance_report.json
  inflating: oracle-emcc-app-1.0.2/dashboards/template_single_em_single_target_type_multi_target_name.json
  inflating: oracle-emcc-app-1.0.2/dashboards/database_configuration_report.json
  inflating: oracle-emcc-app-1.0.2/dashboards/template_single_em.json

We now have to restart Grafana server.

$ sudo systemctl restart grafana-server
$ sudo systemctl status grafana-server
● grafana-server.service - Grafana instance
   Loaded: loaded (/usr/lib/systemd/system/grafana-server.service; disabled; vendor preset: disabled)
  ...
Jul 01 16:21:11 grafana-s02 systemd[1]: Started Grafana instance.
Jul 01 16:21:11 grafana-s02 grafana-server[5675]: t=2020-07-01T16:21:11+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=[::]:3000 protocol=http subUrl= socket=

The EM App for Grafana has been installed, now we need to enable it and start using it. Before we do that we need to also enable it in our EM installation first. There 3 emctl command that we need to execute:

$ emctl set property -name oracle.sysman.db.restfulapi.grafana.enable -value true -sysman_pwd ********
Oracle Enterprise Manager Cloud Control 13c Release 4
Copyright (c) 1996, 2020 Oracle Corporation.  All rights reserved.
Property oracle.sysman.db.restfulapi.grafana.enable has been set to value true for all Management Servers
OMS restart is not required to reflect the new property value

$ emctl set property -name oracle.sysman.db.restfulapi.grafana.executesql.repository.query.enable -value true -sysman_pwd ********
Oracle Enterprise Manager Cloud Control 13c Release 4
Copyright (c) 1996, 2020 Oracle Corporation.  All rights reserved.
Property oracle.sysman.db.restfulapi.grafana.executesql.repository.query.enable has been set to value true for all Management Servers
OMS restart is not required to reflect the new property value

$ emctl set property -name oracle.sysman.db.restfulapi.grafana.executesql.target.query.enable -value true -sysman_pwd ********
Oracle Enterprise Manager Cloud Control 13c Release 4
Copyright (c) 1996, 2020 Oracle Corporation.  All rights reserved.
Property oracle.sysman.db.restfulapi.grafana.executesql.target.query.enable has been set to value true for all Management Servers
OMS restart is not required to reflect the new property value

Login back to the Grafana server and navigate to Configuration -> Plugins. Scroll down and look for Oracle Enterprise Manager App and click on the Plugin. You will see a button to “Enable” the plugin.

Enable Oracle Enterprise Manager App Plugin

Now is time to configure our Grafana datasource to connect to our Oracle Enterprise Manager installation. Navigate to Configuration -> Data Sources. Click on “Add data source” button. Scroll down and look for Oracle Enterprise Manager. Click on that and fill the required parameters:

  • Name
  • URL
  • User (do not use SYSMAN)
  • Password

Now click on “Save & Test” and look for a confirmation.

In order to verify that our Oracle Enterprise Manager App for Grafana is working properly we can open one of the sample dashboards. Let’s navigate to Dashboards -> Manage and click on the “Sample – Database Performance Report” dashboard. You should see something similar as below.

Sample Grafana – EM Dashboard

You can copy these sample dashboards and adjust them for your needs. You may also develop your own dashboards by querying all the Repository Views that EM exposes as part of the extensibility framework.

Hope this provides some guidance on how to deploy the Oracle Enterprise Manager App for Grafana.

Thanks,
Alfredo

Finding the Weblogic Password for the Oracle Enterprise Manager Marketplace Image

Oracle Enterprise Manager (EM) 13.4 is available in the Oracle Cloud Infrastructure Marketplace. This image is very flexible and allows you to choose between different EM Deployment sizes ranging from Simple (Single node) to Large (Multi-node) setups.

I decided to deploy a Simple configuration in order to test it out. The deployment wizard prompts you for several passwords including the Enterprise Manager (sysman), Agent registration, Database password, etc. But it never asks for the Weblogic console password. I first thought that the Weblogic console and Enterprise Manager (sysman) password were going to be the same, but for my surprise they weren’t.

The Marketplace image comes with RU 2 deployed and I need to apply the latest RU 3 to this Oracle EM installation and I must know the Weblogic console username and password in order to accomplish this.

In one of my previous posts, I explained how to apply this RU 3 to Oracle EM.



So, the question now is… What are the username and password for the Weblogic console? Is there any way to change them if I don’t know them?

There’s an MOS note available that explains how to change these passwords when the current password is known.

EM 12c , EM 13c: Steps for Modifying the Password for Weblogic and Nodemanager User Accounts in the Enterprise Manager Cloud Control Installation (Doc ID 1450798.1)

But in this case, I don’t know it. For some reason I remembered that someone posted a way to decrypt this password many years ago when we were running EM 12c. I’m glad that the browser search came back with the result I was looking for. Gokhan Atil posted the method in 2015. Below is the link to his post.



Now is time to follow the steps and find that needed password.

[oracle@oms1 em]$ cd /u01/app/oracle/em/gc_inst_134/user_projects/domains/GCDomain/
[oracle@oms1 GCDomain]$ cat servers/EMGC_ADMINSERVER/security/boot.properties
# Generated by Configuration Wizard on Sat May 09 09:30:41 GMT 2020
username={AES}oK4uyE6/MrBQd+38zr+wSk5y4vTMkQjB19ZOri4sDOI=
password={AES}hUM4t0cbEIJRuluMPxGBN3yGom1M9jwxEhNeGByu+8vTnmbG+xOTR
[oracle@oms1 GCDomain]$ vi recoverpassword.java
#####Insert this content#######
public class recoverpassword {
 public static void main(String[] args)
 {
  System.out.println(
  new weblogic.security.internal.encryption.ClearOrEncryptedService(
  weblogic.security.internal.SerializedSystemIni.getEncryptionService(args[0]
   )).decrypt(args[1]));
  }
}
#####Code ends here#########
[oracle@oms1 GCDomain]$ . bin/setDomainEnv.sh
[oracle@oms1 GCDomain]$ javac recoverpassword.java
[oracle@oms1 GCDomain]$ java -cp $CLASSPATH:. recoverpassword \
> $DOMAIN_HOME {AES}oK4uyE6/MrBQd+38zr+wSk5y4vTMkQjB19ZOri4sDOI=
weblogic
[oracle@oms1 GCDomain]$ java -cp $CLASSPATH:. recoverpassword \
> $DOMAIN_HOME {AES}hUM4t0cbEIJRuluMPxGBN3yGom1M9jwxEhNeGByu+8vTnmbG+xOTR
*************

Voila! Now I have both the username and password for the Weblogic console. Now I can proceed and apply the RU 3 to the OMS instance.

Thanks and happy patching!
Alfredo

DevOps, Ansible and Oracle Enterprise Manager

As IT moves towards virtualization, cloud and hybrid environments, containers, agile developments, etc. Automation and orchestration tools are gaining popularity to build and deliver environments faster and efficiently.

In this post, I want to show how Oracle Enterprise Manager (EM) interacts with 3rd party tools and how this can be used to integrate EM’s functionality with DevOps tools like Ansible or Terraform for configuration management and automation.

Oracle Enterprise Manager (EM) offers a complete tool-set for monitoring, provisioning, upgrade and patching, consolidation, compliance, DBaaS, etc., for the Oracle Database. Oracle Enterprise Manager makes use of Plug-ins and managements packs in order to discover, monitor and manage targets. EM’s functionality can be further extended with connectors and the EM’s extensibility framework in order to interact with 3rd party tools.



You can interact with Oracle Enterprise Manager (EM) in several ways:

  • Oracle Enterprise Manager Console
  • Oracle Enterprise Manager Command Line Interface (EMCLI)
  • Management Repository Views
  • Web Service REST APIs
  • Executing SQL via REST APIs (13.4 only)

We can leverage all the EMCLI commands and REST API’s that Oracle EM exposes with tools like Ansible or Terraform for a complete automation and orchestration solution.

One of the requirements for Oracle EM to monitor and manage a target is to deploy an Oracle Management Agent (OMA) to the machine hosting the target. Many organizations use DevOps tools to deploy virtual machines in an automated fashion. We can integrate the Oracle EM’s agent deployment process with these tools. This helps to drive consistency and reduces the efforts and time that administrators spend deploying Oracle Management Agents.

Let’s deploy an OMA to a Linux host using Ansible and Oracle’s EMCLI interface.

First of all, I need to download and install Ansible in a test server. My test server is a small virtual machine running Oracle Linux 7.

[root@ansible-s02 ~]# yum install -y ansible
Loaded plugins: langpacks, ulninfo
ol7_UEKR5                                                | 2.8 kB     00:00
ol7_addons                                               | 2.8 kB     00:00
ol7_developer                                            | 2.8 kB     00:00
ol7_developer_EPEL                                       | 3.4 kB     00:00
ol7_ksplice                                              | 2.8 kB     00:00
ol7_latest                                               | 3.4 kB     00:00
ol7_oci_included                                         | 2.9 kB     00:00
ol7_optional_latest                                      | 2.8 kB     00:00
ol7_software_collections                                 | 2.8 kB     00:00
(1/19): ol7_UEKR5/x86_64/updateinfo                        |  64 kB   00:00
(2/19): ol7_developer/x86_64/primary_db                    | 544 kB   00:00
(3/19): ol7_developer_EPEL/x86_64/group_gz                 |  87 kB   00:00
(4/19): ol7_developer_EPEL/x86_64/updateinfo               | 6.3 kB   00:00
(5/19): ol7_addons/x86_64/updateinfo                       |  91 kB   00:00
(6/19): ol7_ksplice/updateinfo                             | 5.3 kB   00:00
(7/19): ol7_addons/x86_64/primary_db                       | 153 kB   00:00
(8/19): ol7_latest/x86_64/group_gz                         | 134 kB   00:00
(9/19): ol7_ksplice/primary_db                             | 964 kB   00:00
(10/19): ol7_latest/x86_64/updateinfo                      | 2.9 MB   00:00
(11/19): ol7_developer/x86_64/updateinfo                   | 7.2 kB   00:00
(12/19): ol7_UEKR5/x86_64/primary_db                       | 6.9 MB   00:00
(13/19): ol7_oci_included/x86_64/primary_db                | 211 kB   00:00
(14/19): ol7_optional_latest/x86_64/updateinfo             | 1.0 MB   00:00
(15/19): ol7_software_collections/x86_64/updateinfo        | 8.7 kB   00:00
(16/19): ol7_software_collections/x86_64/primary_db        | 4.9 MB   00:00
(17/19): ol7_developer_EPEL/x86_64/primary_db              |  12 MB   00:00
(18/19): ol7_latest/x86_64/primary_db                      |  24 MB   00:00
(19/19): ol7_optional_latest/x86_64/primary_db             | 4.7 MB   00:00
...
Installed:
  ansible.noarch 0:2.8.4-1.0.1.el7

Dependency Installed:
  python-httplib2.noarch 0:0.9.2-0.1.el7
  python-paramiko.noarch 0:2.1.1-9.el7
  python2-jmespath.noarch 0:0.9.4-1.el7
  python3.x86_64 0:3.6.8-13.0.1.el7
  python3-libs.x86_64 0:3.6.8-13.0.1.el7
  python3-pip.noarch 0:9.0.3-7.el7_8
  python3-setuptools.noarch 0:39.2.0-10.el7
  sshpass.x86_64 0:1.06-1.el7

Complete!

Ansible is now installed in the test machine. The Ansible installion will create a directory under “/etc” that contains the configuration, host inventory and roles.

[root@ansible-s02 ~]# cd /etc/ansible/
[root@ansible-s02 ansible]# ls
ansible.cfg  hosts  roles

Our next step is to edit the host inventory file and add both, our Oracle EM server and the machine where we want to deploy the OMA into. This will look similar to my configuration file below.

# This is the default ansible 'hosts' file.
#
# It should live in /etc/ansible/hosts
#
#   - Comments begin with the '#' character
#   - Blank lines are ignored
#   - Groups of hosts are delimited by [header] elements
#   - You can enter hostnames or ip addresses
#   - A hostname/ip can be a member of multiple groups

# Ex 1: Ungrouped hosts, specify before any group headers.

## green.example.com
## blue.example.com
## 192.168.100.1
## 192.168.100.10

# Ex 2: A collection of hosts belonging to the 'webservers' group

## [webservers]
## alpha.example.org
## beta.example.org
## 192.168.1.100
## 192.168.1.110

# If you have multiple hosts following a pattern you can specify
# them like this:

## www[001:006].example.com

# Ex 3: A collection of database servers in the 'dbservers' group

## [dbservers]
##
## db01.intranet.mydomain.net
## db02.intranet.mydomain.net
## 10.25.1.56
## 10.25.1.57

# Here's another example of host ranges, this time there are no
# leading 0s:

## db-[99:101]-node.example.com


[local]
127.0.0.1

[emserver]
10.0.0.28

[emtarget]
10.0.0.11

I also configured a password-less SSH connectivity between the Ansible test machine and both, the EM’s server and the server where I want to deploy the OMA into. After this, I’m going to verify the connectivity from Ansible to both servers.

[root@ansible-s02 ansible]# ansible emserver -m ping -u oracle
 [WARNING]: Platform linux on host 10.0.0.28 is using the discovered Python
interpreter at /usr/bin/python, but future installation of another Python
interpreter could change this. See https://docs.ansible.com/ansible/2.8/referen
ce_appendices/interpreter_discovery.html for more information.

10.0.0.28 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": false,
    "ping": "pong"
}
[root@ansible-s02 ansible]# ansible emtarget -m ping -u oracle
 [WARNING]: Platform linux on host 10.0.0.11 is using the discovered Python
interpreter at /usr/bin/python, but future installation of another Python
interpreter could change this. See https://docs.ansible.com/ansible/2.8/referen
ce_appendices/interpreter_discovery.html for more information.

10.0.0.11 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python"
    },
    "changed": false,
    "ping": "pong"
}

I see the SUCCESS flag on both of them. Now is time to develop my .yml file with all the necessary steps to deploy the OMA to the host.

Let’s pause for a minute and review all the available options to deploy an OMA to servers. The list below show’s these options:

  • Oracle Enterprise Manager Console
  • Oracle Enterprise Manager Command Line Interface (EMCLI)
  • Silent Mode
    • Using the AgentPull script
    • Using the agentDeploy script
    • Using the RPM file


I decided to use EMCLI to deploy the OMA to a host in this example. The .yml file to deploy the OMA looks like:

- hosts: emtarget
  vars:
        emagent_port: 3874
        emagent_base: /u01/app/oracle/product/agent134c
  tasks:
    - name: check emagent port
      wait_for: port={{ emagent_port }} state=stopped timeout=1
    - name: creating base directory
      file: path={{ emagent_base }} state=directory owner="oracle" group="oinstall"
      tags: install

- hosts: emserver
  vars:
        emagent_port: 3874
        emagent_base: /u01/app/oracle/product/agent134c
        emagent_hostname: emtarget.oracle.com
        emagent_platform_id: 226
  tasks:
    - name: execute emcli
      command: "/u01/app/oracle/product/omshome/bin/emcli submit_add_host -host_names={{ emagent_hostname }} -platform={{ emagent_platform_id }} -installation_base_directory={{ emagent_base }} -credential_name=\"HOST_ORACLE\" -port={{ emagent_port }} -wait_for_completion"
      register: result

    - name: show results
      debug:
        var: result.stdout

Now is time to execute our Ansible Playbook.

[root@ansible-s02 ansible]# ansible-playbook emcli_deploy_agent.yml -u oracle

PLAY [emtarget] *************************************************************************************************************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] ******************************************************************************************************************************************************************************************************************************************************************************************************
ok: [10.0.0.11]

TASK [check emagent port] ***************************************************************************************************************************************************************************************************************************************************************************************************
ok: [10.0.0.11]

TASK [creating base directory] **********************************************************************************************************************************************************************************************************************************************************************************************
ok: [10.0.0.11]

PLAY [emserver] *************************************************************************************************************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] ******************************************************************************************************************************************************************************************************************************************************************************************************
ok: [10.0.0.28]

TASK [execute emcli] ********************************************************************************************************************************************************************************************************************************************************************************************************
changed: [10.0.0.28]

TASK [show results] *********************************************************************************************************************************************************************************************************************************************************************************************************
ok: [10.0.0.28] => {
    "result.stdout": "Session Name : ADD_HOST_SYSMAN_Jun_26_2020_11:54:55_AM_EDT\nOverAll Status : Agent Deployment Succeeded\n\nHost                 Platform Name  Initialization  Remote Prerequisite  Agent Deployment  Error\nemtarget.oracle.com  Linux x86-64   Succeeded       Succeeded            Succeeded       "
}

PLAY RECAP ******************************************************************************************************************************************************************************************************************************************************************************************************************
10.0.0.11                  : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
10.0.0.28                  : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Now let’s verify our agent deployment. I’m going to login to the Oracle Enterprise Manager Console and I should be able to see the details of the deployment.

Oracle Enterprise Manager Agent Deployment Dashboard

Here you go! The OMA was successfully deployed to my target host. I now need to manually execute the “root.sh” script, this because I didn’t specify a privileged credential during the deployment.

Hope this helps to demonstrate the flexibility of Oracle’s EM and also as an example on how to integrate it’s functionality with DevOps tools like Ansible. This is not by any means the best way to deploy an agent for your particular environment but rather an example of how to integrate EM with Ansible.

Thanks,
Alfredo