EM12c Release 4: Job System Easter Eggs – Part 1: New Progress Tracking UI

Tags

Automation, em12c, job system

So you just installed a new EM12c R4 environment or upgraded your existing EM environment to EM12c R4. Post upgrade you go to the Job System activity page (via Enterprise->Job->Activity menu) and view the progress details  of a job. Well nothing seems to have changed, its the same UI, the same multi-page drill down to view step output, same no. of clicks, etc. Wrong! In this two part blog post, i talk about two Job System Easter Eggs (hidden features) that most of you will find interesting. These are:

1. New Job progress tracking UI
2. Import/Export of job definitions
   

So before i go any further, let me address the issue of why are these features hidden? As we were building these features, we realized that we would not be ready to ship the desired quality of code by the set dates. Hence, instead of removing the code, it was decided to ship it in a disabled state so as not to impact customers, but still allowing a brave few to experiment with it and provide valuable feedback.

1.  New Job Progress Tracking UI

The job system UI hasn’t changed much since its introduction almost 10 years ago. It is a daunting task to update all the job system related UIs in a single release, and hence we decided to take a piecemeal approach instead. In the first installment, we have revamped the job progress tracking page.

The current UI, as shown above, while being very functional, is also very laborious. Multiple clicks and drill downs are required to view the step output for a particular target. Also, any click leads to complete page refresh, which leads to wastage of time and resources. The new UI tries to address all these concerns. It is a single page UI, which means no matter where you click, you never leave the page and thus never lose context of the target or step you where in. It also significantly reduces the no. of clicks required to complete the same task as in the current UI. So lets take a look at this new UI.

First, as i mentioned earlier, you need to enable this UI. To do this, you need to run the following emctl command from any of the OMS:

./emctl set property -name oracle.sysman.core.jobs.ui.useAdfExecutionUi -value true

This command will prompt for the sysman password, and then will enable the new UI.

NOTE: This command does not require a restart of the OMS. Once run, the new UI will be enabled for all user across all OMSes.

Now revisit the job progress tracking page from before. You will be directed to the new UI.

There are in all 6 key regions on this new single page job progress tracking UI. Starting from top left, these are:

1. Job Run Actions – These are actions that can be performed on the job run like suspend resume, retry, stop, edit, etc.
2. Executions – This region displays all the executions in the job run. An execution, in most cases, represents a single target and hence runs independently from other executions. This region thus shows the progress and status of all executions in a single view. The best part of this region is the column titled ‘Execution Time’. The cigar chart in this column represents two things – one, the duration of the execution, and two, the difference in start times. The visual representation helps in identifying runaway executions, or just compare execution times across different targets. The Actions menu allows various options like start, stop, debug, delete, etc.
3. Execution Summary – Clicking on an execution in the above region, paints the area on the right. This specific region shows execution summary with information like status, start & end date, execution id, command, etc
4. Execution Steps – This region lists the steps that make up the execution.
5. Step Output – Clicking on a step from the above region, paints this region. This shows the details of the step. This includes the step output and the ability to download it to a text file.
6. Page Options – We imagine that learning any new UI takes time, and hence this final region provides the option to switch between the new and the classic view. Additionally, this also allows you to set the auto refresh rate for the page.

Essentially, considering that jobs have two levels – executions and steps, we have experimented with a multi-master style layout. EM has never used such a layout and hence there were concerns raised when we chose to do so.

Master 1 (region 2) -> Detail 1 (regions 3, 4, & 5)

Master 2 (region 4) -> Detail 2 (region 5)

In summary, with this new UI, we have been able to significantly reduce the no. of clicks required to track job progress and drill into details. We have also been able to show all relevant information in a single page, thus avoiding unnecessary page redirection and reloads. I would love to hear from you if this experiment has paid off and if you find this new UI useful.

In the next part of this blog i talk about the new emcli verbs to import and export job definitions across EM environments. This has been a long standing enhancement request, and we are quite excited about our efforts.

Advertisement

Oracle Private Database Cloud REST APIs

Tags

apis, Automation, database as a service, dbaas, em12c, private cloud

While we spend a lot of time developing GUI for our private cloud features – like the self service portal, setup screens, etc, a large percentage of our customers use these features via our APIs, both EMCLI and REST based.

Q: So why would anyone not use the out of the box cloud (self service) portal and use the APIs instead?

A: Well, there are many reasons:

1. You already have a custom cloud portal that you use to deploy non-Oracle products and now you would like Oracle products to be part of it
2. You want cloud related actions to be part of a larger orchestration flow. For example, prior to provisioning, you want to integrate an approval workflow, and then post-provision update your asset management system with the service details.
3. You want to integrate the cloud actions with our IT service desk or ticketing systems
4. Integrate actions like cloning of databases (Snap Clone), or provisioning of middleware and applications with your continuous integration or devops process.

I am sure there are more reasons, but this list should do for now.

Q: What kind of APIs does EM12c expose?

A: Typically, we identify two key roles in the Private Cloud space:

1. Service Providers – This role represents the various administrators like DBAs, sysadmins, storage admins, etc that help setup and manage the private cloud infrastructure. EM12c provides EMCLI verbs and RESTful APIs for all actions that can be performed by this role.
2. Service Consumers – This role represents the users who request and consume cloud based services like databases, VMs, middleware servers, etc. This role can access the cloud services from either the out-of-the-box cloud portal, or using RESTful APIs.

This blog post is focused on the the RESTful APIs provided for Service Consumers. The best way to understand and learn about these APIs is to see them in action. In the video below, i explain the various concepts (note i assume that you are already familiar with EM12c DBaaS concepts, if not check the references section), demo the cloud portal, and then perform the same actions using RESTful APIs. The demo is approx. 23 minutes long, so grab a drink before you get started.

The Python script used to demo the APIs can be found here. Note: i am posting this script as sample implementation, so there is no formal support available for it. Also, the demo was recorded on the latest version of EM – EM12c R4 Plugin Update 1.

Finally, I am often asked about customer examples. Till date, i have seen customers use our REST APIs from Chef recipes, shell scripts using the curl command, from 3rd party tools like Service Now and VMware vCAC, custom Java based applications, etc. Unfortunately, i am not an expert on any of these 3rd party tools, so i probably cannot tell you how the integration was done, but i would be more than happy to help you with the EM REST APIs.

Hopefully this was a good introduction to our EM12c Database as a Service (Private Database Cloud) REST APIs. I encourage you to play with the APIs and post code examples for the benefit of other users.

References:

Screenwatch: Oracle Enterprise Manager 12c: Build Service Catalog with DBaaS

Documentation: Cloud Administration Guide, REST API Chapter

Editing EM12c Jobs in Bulk

Tags

em12c, emcli, job system

I recently received requests for suggestions on how to edit EM12c jobs in bulk. The two examples that were presented to me were:

1. Change the start time of 70+ jobs
2. Change the oracle home path in 80+ RMAN script jobs

So how do we do this?

Option 1: Log into EM12c, navigate to each and every job, click edit and make the desired changes. This of course gets boring after the 4th job, not to mention the potential for developing carpel tunnel syndrome.

Option 2: The other choice is to scan the EM repository tables and try to update the correct columns. Unfortunately, the EM schema is not that simple to follow and your changes to the repository may not take effect as they haven’t updated all relevant references or have not been called from the correct set of APIs. BTW, this also violates our recommendation to never make changes to the EM schema without specific instructions from Oracle support or development, and has the potential of messing up your repository.

Option 3: This makes use of the job system emcli verbs, and is the only viable solution to the problem. The process would be as follows:

1. Use emcli verb “describe_job” to dump all required jobs into separate properties files (I am assuming you have list of job names that needs to be edited)
2. Mass-edit the property files to change the start date or any other parameters you like
3. Create new jobs using these property files with slightly different names using the emcli verb ”create_job”
4. Once satisfied with newly created jobs, remove older jobs using “delete_job” emcli verb

Documentation for these verbs can be found here.

Now you could argue that option 3 is not perfect either, and i would give you credit for making that argument. Hence an ER has been filed to provide a formal ‘edit_jobs’ verb which will allow you to edit jobs via emcli either in bulk or in a loop. Until then, hopefully this serves as a good work around.

Monitoring NFS mounted file systems using EM12c

Tags

em12c, host management, monitoring, nfs, oracle enterprise manager

A customer recently asked me how they could monitor and alert against all the NFS mounted file systems across their datacenter. Here is a quick guide to do the same.

A. Lets say for starters, all you want to do is view details of all NFS mounted file systems on a single Host target.

Solution:

Navigate to the Host target home page. Access the menu item Host->Storage Details. Switch to the file systems tab and filter the list by NFS. You can now happily view used and total sizes of your file systems.

B. Now you want to list details of all your NFS mounted file systems across all Host targets.

Solution:

The best way to do this would be to create a report using the published repository view called ‘MGMT$STORAGE_REPORT_NFS‘. The view is rather straight forward and easy to use.

C. While the first two cases were more focused on viewing the information, lets say now you want to set a threshold and actively monitor these NFS file systems.

Solution:

All the data we have viewed so far is collected as a configuration metric, and hence its collection frequency is 24 hrs and you cannot set thresholds against it directly. So for the purposes of active monitoring, we will navigate to the All Metrics page (Host->Monitoring->All Metrics) on the host target and take a look at ‘Filesystem Space Available (%)‘ metric. This metric will allow you to set thresholds and get alerts on file system space usage. Unfortunately, when you view the output of this metric, you will not find any of your NFS mounted file systems. This is because the collection for NFS mounted file systems is disabled by default. To enable this:

1. Go to $AGENT_HOME/sysman/config/emd.properties on the desired set of hosts

2. Add the property EM_MONITOR_ALL_DISKS=true

3. Run $AGENT_HOME/bin/emctl reload agent

Wait about 15 minutes, since that is the default collection interval. Refresh the page and Viola! all your NFS mounted file systems will appear on the all metrics page. Any thresholds set for this metric will now automatically apply to your NFS mounted file systems.

Finally, making the changes on a single agent is simple enough, but doing this manually across 100s of agents is rather tedious. To automate this, go to the Agents page (Setup->Manage Cloud Control->Agents), select the desired agents and click on the Properties button in the toolbar. This will launch a job submission wizard where you can select additional agents. In the parameters tab, find the EM_MONITOR_ALL_DISKS property and set its value to ‘true’, click submit. A job is submitted which takes only a fraction of a second to finish, and takes care of applying the changes to all agents and activating them. More details on this job can be found here.

Enjoy!

– Adeesh Fulay (@AdeeshF)

Download Urls for Self Update Entities in Offline Mode

Tags

em12c, offline mode, oracle enterprise manager, plugins, self update

Self update in the online mode is fairly easy to use. I mean it is hard to beat a user interface that simply has three primary actions –

• Check Update – check for latest updates available for download. This also happens automatically every day, so there is no explicit need to use this button.
• Download – download your favorite update (plug-in, agent, connector, etc) to a local directory in EM
• Apply – apply the downloaded update to EM

Here is a good overview video for Self Update in Online Mode.

Unfortunately, there are a lot of customers (Federal, Banking, Financial, etc) who are not allowed to use the Online mode due to various security reasons. We did anticipate this and thus designed a complete offline experience for such customers. In a nutshell, the process can be described as follows:

1. Assuming, your MOS connection settings are set to OFFLINE, a user when clicks on the ‘Check Updates’ button, is shown the download url that can be used to download the metadata required to populate the Self Update UI
2. Once this archive is downloaded, you can run ’emcli import_update_catalog’ from an agent host or from the OMS to import the metadata
3. Now you browse through the table of entities, choose your favorite one, and click on the download button. This is just like you would do in the online mode, with the exception that instead of download being kicked off, you would again see a popup with the download url for the payload along with instructions on how to apply them.
4. Once you download the new archive, run emcli import_update to import the payload into self update from one of the agents or from the OMS host
5. Finally, we go back to the UI, and apply the entities

While the above looks quite involved, in reality it should take you less than 5 mins (minus the download time) to complete the entire process. This is not bad.

A challenge with the above approach thought is that steps 3-5 have to be repeated for every entity and this can be time consuming. Especially for step 3, you have to click on each interesting entity to determine its download url. While we are working on formally addressing this, here is a quick hack to make your life easier.

In EM12c R4, we introduced an internal view that contains all the relevant information for self update entities and their download url. This view is called gc$selfupdate_entity. That fact that this view is internal means (the usual disclaimers) – firstly, the view has been created for internal consumption by EM, and not by customers, and secondly, we reserve the right to modify this view at any time.

Here are a few useful queries:

1. Every release, whether it is platform or a plug-in release, brings a slew of new entities. Majority of these entities are plug-ins. The following query lists the download urls of all new plug-ins that are in the ‘Available’ state:

SELECT su2.attr4_label,
su2.version,
su2.entity_id,
su2.url
FROM gc$selfupdate_entity su2,
(SELECT su.attr4_label,
MAX(su.version) max_version
FROM gc$selfupdate_entity su
WHERE et_name=’core_emplugin’
AND su.status=’Available’
GROUP BY attr4_label
) max_su
WHERE su2.attr4_label = max_su.attr4_label
AND su2.version       = max_su.max_version
ORDER BY 1;

2. Similarly, this query lists the download urls for all new agents in the ‘Available’ state:

SELECT su2.attr3_label,
su2.version,
su2.entity_id,
su2.url
FROM gc$selfupdate_entity su2,
(SELECT su.attr3_label,
MAX(su.version) max_version
FROM gc$selfupdate_entity su
WHERE et_name=’core_agent_core_image’
AND su.status=’Available’
GROUP BY su.attr3_label
) max_su
WHERE su2.attr3_label = max_su.attr3_label
AND su2.version       = max_su.max_version
ORDER BY 1;

—

Adeesh Fulay (@AdeeshF)