13c agent oracle.sysman.gcagent.task.TaskFutureImpl$WrappedTask:621

We found have error at A server with a 13c's agent and the number of occurrences is frequent.

we found error informations from  the gcagent_erro.log file.


oracle.sysman.gcagent.task.TaskZombieException: task declared as a zombie
    at oracle.sysman.gcagent.task.TaskFutureImpl$WrappedTask.accountedCall(TaskFutureImpl.java:621)
    at oracle.sysman.gcagent.task.TaskFutureImpl$WrappedTask.call(TaskFutureImpl.java:644)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at oracle.sysman.gcagent.task.TaskFutureImpl.run1(TaskFutureImpl.java:373)
    at oracle.sysman.gcagent.task.TaskFutureImpl.run(TaskFutureImpl.java:330)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at oracle.sysman.gcagent.task.executor.TrackThreadFactory$1.run(TrackThreadFactory.java:54)
    atoracle.sysman.gcagent.util.system.GCAThread$RunnableWrapper.run(GCAThread.java:198)
    at java.lang.Thread.run(Thread.java:745)
2017-02-16 15:30:01,734 [2713889:GC.Executor.124190
(oracle_database:hpaypr1:Response)] ERROR - oracle_database:hpaypr1:Response
oracle.sysman.gcagent.task.TaskZombieException: task declared as a zombie
    at oracle.sysman.gcagent.task.TaskFutureImpl$WrappedTask.accountedCall(TaskFutureImpl.java:621)
    at oracle.sysman.gcagent.task.TaskFutureImpl$WrappedTask.call(TaskFutureImpl.java:644)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at oracle.sysman.gcagent.task.TaskFutureImpl.run1(TaskFutureImpl.java:373)
    at oracle.sysman.gcagent.task.TaskFutureImpl.run(TaskFutureImpl.java:330)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at oracle.sysman.gcagent.task.executor.TrackThreadFactory$1.run(TrackThreadFactory.java:54)
    at oracle.sysman.gcagent.util.system.GCAThread$RunnableWrapper.run(GCAThread.java:198)
    at java.lang.Thread.run(Thread.java:745)

CAUSE:
When an agent task such as collecting metrics is running more than the expected time, the process is marked as Zombie.
This generates an Incident in order to let the Enterprise Manager (EM) Administrator know that a task has generated a zombie process.
The zombie process is a valid mechanism of the agent.
A parameter to eliminate Incident creation for Zombies needs to be added to the $AGENT_INST/sysman/config/emd.properties. 
This parameter was not documented; and a bug was filed.

TECHNICAL IMPACT
1.Zombie processes and Incidents created. 
2.Agent can try to restart
 
RELATED ISSUES (bugs, forums, RFAs)
similar to bug : Bug 22221528 – Zombie process seen during DB Metric collection

SOLUTION:
In order to avoid the Zombie Process Alerts/Incidents, the following parameter can be added to the $AGENT_INST/sysman/config/emd.properties:

_zombieCreateIncident=false 

1. Edit the file emd.properties
2. Add the parameter : _zombieCreateIncident=false
3. emctl stop agent
4. emctl start agent

Other parameter reference:
_canceledThreadWait == > The amount of time before a canceled thread is considered a 'zombie' thread
_zombieSuspensions ==> Whether to indefinitely suspend a zombie task
_zombieThreadPercentThreshold ==> The number of 'zombie' threads (as a percent of the REGULAR thread pool) before a bounce will be started; set to zero to disable auto restarts
Note: Setting these parameters , agent restart due to zombies would reduce/stop and incident  dump shall get created
_zombieCreateIncident==>Whether to create an incident for the zombie abort

13cR2 Modify OMS HTTPS and HTTP Console And Upload Ports After 13c Cloud Control Installation

13cR2 Modify OMS HTTPS and HTTP Console And Upload Ports After 13c Cloud Control Installation

Cause:
The port is occupied, "em13c" directory had moved,but the process still active:

[oracle@trsen:/u01/app/em13cR2/middle/bin]$netstat -apn | grep 7802
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 :::7802                     :::*                        LISTEN      6986/httpd.worker   
[oracle@trsen:/u01/app/em13cR2/middle/bin]$ps -aux | grep 6986
Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.8/FAQ
oracle    3893  0.0  5.4 4016748 669868 ?      Ss   Nov10   0:40 ora_p006_EMREP
oracle    6986  0.0  0.0 1998880  644 ?        Sl   Nov11   0:07 /u01/app/em13c/middleware/ohs/bin/httpd.worker -k start -f /u01/app/em13c/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/instances/ohs1/httpd.conf

Solution:
1.lists the current value of the https/http console/upload ports informations are followed:
Now port for oms:
HTTP Console Port          : 7790
HTTPS Console Port         : 7803
HTTP Upload Port           : 4890
HTTPS Upload Port          : 4904

we modify port to new port:
HTTP Console Port          : 7788
HTTPS Console Port         : 7802
HTTP Upload Port           : 4889
HTTPS Upload Port          : 4903

OMS property of ports:
HTTPS CONSOLE PORT:oracle.sysman.emSDK.svlt.EMConsoleServerHTTPSPort
HTTPS UPLOAD PORT:oracle.sysman.emSDK.svlt.ConsoleServerHTTPSPort
HTTP CONSOLE PORT:oracle.sysman.emSDK.svlt.EMConsoleServerPort
HTTP UPLOAD PORT:oracle.sysman.emSDK.svlt.ConsoleServerPort

Changing HTTPS console port:
1. Verify the existing https console port:

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl get property -name  oracle.sysman.emSDK.svlt.EMConsoleServerHTTPSPort -sysman_pwd em13c09
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Value for property oracle.sysman.emSDK.svlt.EMConsoleServerHTTPSPort for oms trsen.em13c.cn:4890_Management_Service is 7803

2. Stop the OMS

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl stop oms
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Stopping Oracle Management Server...
Oracle Management Server Successfully Stopped
Oracle Management Server is Down
JVMD Engine is Down

3.Change the https console port

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl set property -name oracle.sysman.emSDK.svlt.EMConsoleServerHTTPSPort -value 7802 -sysman_pwd em13c09
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Property oracle.sysman.emSDK.svlt.EMConsoleServerHTTPSPort for oms trsen.em13c.cn:4890_Management_Service has been set to value 7802
OMS restart is required to reflect the new property value

4.Take a backup of file and update ssl.conf and emgc.properties :

[oracle@trsen:/u01/app/em13cR2/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/ohs1]$cat ssl.conf | grep 7803
Listen 7803 #OHS_SSL_PORT
<VirtualHost *:7803> #OHS_SSL_VH

[oracle@trsen:/u01/app/em13cR2/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/ohs1]$cat ssl.conf | grep 7802
Listen 7802 #OHS_SSL_PORT
<VirtualHost *:7802> #OHS_SSL_VH

[oracle@trsen:/u01/app/em13cR2/gc_inst/em/EMGC_OMS1]$cat emgc.properties | grep 7803
EM_CONSOLE_HTTPS_PORT=7803

[oracle@trsen:/u01/app/em13cR2/gc_inst/em/EMGC_OMS1]$cat emgc.properties | grep 7802
EM_CONSOLE_HTTPS_PORT=7802

5.restart oms:

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl stop oms -all -force
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Stopping Oracle Management Server...
WebTier Successfully Stopped
Oracle Management Server Already Stopped
Oracle Management Server is Down
JVMD Engine is Down
BI Publisher is disabled, to enable BI Publisher on this host, use the 'emctl config oms -enable_bip' command
Stopping BI Publisher Server...
BI Publisher Server Already Stopped
BI Publisher is disabled, to enable BI Publisher on this host, use the 'emctl config oms -enable_bip' command
AdminServer Successfully Stopped
BI Publisher Server is Down
BI Publisher is disabled, to enable BI Publisher on this host, use the 'emctl config oms -enable_bip' command

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl start oms
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Starting Oracle Management Server...
WebTier Successfully Started
Oracle Management Server Successfully Started
Oracle Management Server is Up
JVMD Engine is Up

Changing HTTPS Upload Port
1. Verify the OMS HTTPS upload port:

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl get property -name oracle.sysman.emSDK.svlt.ConsoleServerHTTPSPort -sysman_pwd em13c09
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Value for property oracle.sysman.emSDK.svlt.ConsoleServerHTTPSPort for oms trsen.em13c.cn:4890_Management_Service is 4904

2. Modify the HTTPS upload port to new value:

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl set property -name oracle.sysman.emSDK.svlt.ConsoleServerHTTPSPort -value 4903 -sysman_pwd em13c09
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Property oracle.sysman.emSDK.svlt.ConsoleServerHTTPSPort for oms trsen.em13c.cn:4890_Management_Service has been set to value 4903
OMS restart is required to reflect the new property value

3. Take a backup of the file and update emgc.properties and httpd_em.conf(have two places) file:

[oracle@trsen:/u01/app/em13cR2/gc_inst/em/EMGC_OMS1]$cat emgc.properties | grep 4904
EM_UPLOAD_HTTPS_PORT=4904
[oracle@trsen:/u01/app/em13cR2/gc_inst/em/EMGC_OMS1]$cat emgc.properties | grep 4903
EM_UPLOAD_HTTPS_PORT=4903

[oracle@trsen:/u01/app/em13cR2/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/instances/ohs1/moduleconf]$cat httpd_em.conf | grep 4904
    Listen 4904
    <VirtualHost *:4904>
[oracle@trsen:/u01/app/em13cR2/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/instances/ohs1/moduleconf]$cat httpd_em.conf | grep 4903
    Listen 4903
    <VirtualHost *:4903>

[oracle@trsen:/u01/app/em13cR2/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/ohs1/moduleconf]$cat httpd_em.conf | grep 4904
    Listen 4904
    <VirtualHost *:4904>
[oracle@trsen:/u01/app/em13cR2/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/ohs1/moduleconf]$cat httpd_em.conf | grep 4903
    Listen 4903
    <VirtualHost *:4903>

4. Restart OMS:

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl stop oms -force -all
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Stopping Oracle Management Server...
WebTier Successfully Stopped
Oracle Management Server Successfully Stopped
Oracle Management Server is Down
JVMD Engine is Down
BI Publisher is disabled, to enable BI Publisher on this host, use the 'emctl config oms -enable_bip' command
Stopping BI Publisher Server...
BI Publisher Server Already Stopped
BI Publisher is disabled, to enable BI Publisher on this host, use the 'emctl config oms -enable_bip' command
AdminServer Successfully Stopped
BI Publisher Server is Down
BI Publisher is disabled, to enable BI Publisher on this host, use the 'emctl config oms -enable_bip' command

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl start oms
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Starting Oracle Management Server...
WebTier Successfully Started
Oracle Management Server Successfully Started
Oracle Management Server is Up
JVMD Engine is Up

NOTE:
If agents are configured with https upload port then perform the following on agent machines to reflect the new OMS https upload port:
1. Take a backup of <AGENT_INST_HOME>/sysman/config/emd.properties file
2. Edit the <AGENT_INST_HOME>/sysman/config/emd.properties file as below:
REPOSITORY_URL=https://omsmachine.domain.com:4903/empbs/upload
emdWalletSrcUrl=https://omsmachine.domain.com:4903/em/wallets/emd
3. Restart the agent:
<AGENT_INST_HOME>/bin>./emctl stop agent
<AGENT_INST_HOME>/bin>./emctl start agent

Changing HTTP console port
1. Verify the OMS HTTP console port:

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl get property -name oracle.sysman.emSDK.svlt.EMConsoleServerPort -sysman_pwd em13c09
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Value for property oracle.sysman.emSDK.svlt.EMConsoleServerPort for oms trsen.em13c.cn:4890_Management_Service is 7790

2. Modify the HTTPS upload port to new value:

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl set property -name oracle.sysman.emSDK.svlt.EMConsoleServerPort -value 7788 -sysman_pwd em13c09
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Property oracle.sysman.emSDK.svlt.EMConsoleServerPort for oms trsen.em13c.cn:4890_Management_Service has been set to value 7788
OMS restart is required to reflect the new property value

3. Take a backup of the file and update emgc.properties and httpd.conf file:

[oracle@trsen:/u01/app/em13cR2/gc_inst/em/EMGC_OMS1]$cat emgc.properties | grep 7790
EM_CONSOLE_HTTP_PORT=7790 
[oracle@trsen:/u01/app/em13cR2/gc_inst/em/EMGC_OMS1]$cat emgc.properties | grep 7788
EM_CONSOLE_HTTP_PORT=7788

[oracle@trsen:/u01/app/em13cR2/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/ohs1]$cat httpd.conf | grep 7790
Listen 7790   #OHS_LISTEN_PORT
[oracle@trsen:/u01/app/em13cR2/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/ohs1]$cat httpd.conf | grep 7788
Listen 7788   #OHS_LISTEN_PORT

4. Restart OMS:

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl stop oms -all -force
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Stopping Oracle Management Server...
WebTier Successfully Stopped
Oracle Management Server Successfully Stopped
Oracle Management Server is Down
JVMD Engine is Down
BI Publisher is disabled, to enable BI Publisher on this host, use the 'emctl config oms -enable_bip' command
Stopping BI Publisher Server...
BI Publisher Server Already Stopped
BI Publisher is disabled, to enable BI Publisher on this host, use the 'emctl config oms -enable_bip' command
AdminServer Successfully Stopped
BI Publisher Server is Down
BI Publisher is disabled, to enable BI Publisher on this host, use the 'emctl config oms -enable_bip' command

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl start oms
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Starting Oracle Management Server...
WebTier Successfully Started
Oracle Management Server Successfully Started
Oracle Management Server is Up
JVMD Engine is Up

Changing HTTP upload port
1. Verify the OMS HTTP upload port:

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl get property -name oracle.sysman.emSDK.svlt.ConsoleServerPort -sysman_pwd em13c09
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Value for property oracle.sysman.emSDK.svlt.ConsoleServerPort for oms trsen.em13c.cn:4890_Management_Service is 4890

2. Modify the HTTP upload port to new value:

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl set property -name oracle.sysman.emSDK.svlt.ConsoleServerPort -value 4889 -sysman_pwd em13c09
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Property oracle.sysman.emSDK.svlt.ConsoleServerPort for oms trsen.em13c.cn:4890_Management_Service has been set to value 4889
OMS restart is required to reflect the new property value

3. Take a backup of the file and update /u02/em13c/gc_inst/em/EMGC_OMS1/emgc.properties file:

[oracle@trsen:/u01/app/em13cR2/gc_inst/em/EMGC_OMS1]$cat emgc.properties | grep 4890
oracle.sysman.emSDK.svlt.ConsoleServerName=trsen.em13c.cn\:4890_Management_Service
EM_UPLOAD_HTTP_PORT=4890
[oracle@trsen:/u01/app/em13cR2/gc_inst/em/EMGC_OMS1]$cat emgc.properties  | grep 4889
oracle.sysman.emSDK.svlt.ConsoleServerName=trsen.em13c.cn\:4889_Management_Service
EM_UPLOAD_HTTP_PORT=4889

[oracle@trsen:/u01/app/em13cR2/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/ohs1/moduleconf]$cat httpd_em.conf | grep 4890
Listen 4890
<VirtualHost *:4890>
[oracle@trsen:/u01/app/em13cR2/gc_inst/user_projects/domains/GCDomain/config/fmwconfig/components/OHS/ohs1/moduleconf]$cat httpd_em.conf | grep 4889
Listen 4889
<VirtualHost *:4889>

4. Restart OMS:

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl stop oms -all -force
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Stopping Oracle Management Server...
WebTier Successfully Stopped
Oracle Management Server Successfully Stopped
Oracle Management Server is Down
JVMD Engine is Down
BI Publisher is disabled, to enable BI Publisher on this host, use the 'emctl config oms -enable_bip' command
Stopping BI Publisher Server...
BI Publisher Server Already Stopped
BI Publisher is disabled, to enable BI Publisher on this host, use the 'emctl config oms -enable_bip' command
AdminServer Successfully Stopped
BI Publisher Server is Down
BI Publisher is disabled, to enable BI Publisher on this host, use the 'emctl config oms -enable_bip' command

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl start oms
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Starting Oracle Management Server...
WebTier Successfully Started
Oracle Management Server Successfully Started
Oracle Management Server is Up
JVMD Engine is Up

NOTE:
If agents are configured with http upload port then perform the following on agent machines to reflect the new OMS http upload port:
1. Take a backup of <AGENT_INST_HOME>/sysman/config/emd.properties file
2. Edit the <AGENT_INST_HOME>/sysman/config/emd.properties file as below:
REPOSITORY_URL=http://omsmachine.domain.com:4889/empbs/upload
emdWalletSrcUrl=http://omsmachine.domain.com:4889/em/wallets/emd
3. Restart the agent:
<AGENT_INST_HOME>/bin>./emctl stop agent
<AGENT_INST_HOME>/bin>./emctl start agent

Verify OMS port informations:

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl status oms -details -sysman_pwd em13c09
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Console Server Host        : trsen.em13c.cn
HTTP Console Port          : 7788
HTTPS Console Port         : 7802
HTTP Upload Port           : 4889
HTTPS Upload Port          : 4903
EM Instance Home           : /u01/app/em13cR2/gc_inst/em/EMGC_OMS1
OMS Log Directory Location : /u01/app/em13cR2/gc_inst/em/EMGC_OMS1/sysman/log
OMS is not configured with SLB or virtual hostname
Agent Upload is unlocked.
OMS Console is unlocked.
Active CA ID: 1
Console URL: https://trsen.em13c.cn:7802/em
Upload URL: https://trsen.em13c.cn:4903/empbs/upload

WLS Domain Information
Domain Name            : GCDomain
Admin Server Host      : trsen.em13c.cn
Admin Server HTTPS Port: 7103
Admin Server is RUNNING

Oracle Management Server Information
Managed Server Instance Name: EMGC_OMS1
Oracle Management Server Instance Host: trsen.em13c.cn
WebTier is Up
Oracle Management Server is Up
JVMD Engine is Up

BI Publisher Server Information
BI Publisher Managed Server Name: BIP
BI Publisher Server is Down
BI Publisher is disabled, to enable BI Publisher on this host, use the 'emctl config oms -enable_bip' command

The error may occur when "emctl status oms -details":

2016-11-14 13:22:12,507 [main] ERROR oms.PrintOMSSetupInfo main.343 – Failed to find OMS details: null
java.lang.NullPointerException
    at oracle.sysman.emctl.util.EmctlUtil.getOMSSetupInfo(EmctlUtil.java:978)
    at oracle.sysman.emctl.oms.PrintOMSSetupInfo.main(PrintOMSSetupInfo.java:155)

solution:

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl secure oms
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Securing OMS... Started.
Enter Enterprise Manager Root (SYSMAN) Password : 
Enter Agent Registration Password : 
Securing OMS... Successful
Restart OMS

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl stop oms -all -force
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Stopping Oracle Management Server...
WebTier Successfully Stopped
Oracle Management Server Successfully Stopped
Oracle Management Server is Down
JVMD Engine is Down
BI Publisher is disabled, to enable BI Publisher on this host, use the 'emctl config oms -enable_bip' command
Stopping BI Publisher Server...
BI Publisher Server Already Stopped
BI Publisher is disabled, to enable BI Publisher on this host, use the 'emctl config oms -enable_bip' command
AdminServer Successfully Stopped
BI Publisher Server is Down
BI Publisher is disabled, to enable BI Publisher on this host, use the 'emctl config oms -enable_bip' command
[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl start oms
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.

[oracle@trsen:/u01/app/em13cR2/middle/bin]$./emctl start oms
Oracle Enterprise Manager Cloud Control 13c Release 2  
Copyright (c) 1996, 2016 Oracle Corporation.  All rights reserved.
Starting Oracle Management Server...
WebTier Successfully Started
Oracle Management Server Successfully Started
Oracle Management Server is Up
JVMD Engine is Up

ORACLE 12C Create PDB’s Techniques.

The techniques that you can use to create a PDB.
1.coping
a.copying files from the seed
b.cloning a pdb(locally,remotely(from a pdb or from a non-cdb)
2.plugging in
a.plugging in an unplugged pdb
b.plugging in a non-cdb as a pdb

一.Using the seed pdb to creating a pdb.
1.my database have to pdbs including:PDBEMREP and PDB$SEED

SYS@EMREP>select pdb_id,pdb_name,dbid,status from dba_pdbs;
    PDB_ID PDB_NAME                      DBID STATUS
---------- ---------------------------------------- ---------- ------------------
     3 PDBEMREP                    3183988881 NORMAL
     2 PDB$SEED                    4121219803 NORMAL

2.In SQL*Plus, ensure that the current container is the root

SYS@EMREP>SELECT SYS_CONTEXT ('USERENV', 'CON_NAME') FROM DUAL;
SYS_CONTEXT('USERENV','CON_NAME')
-------------------------------------------------------------------
CDB$ROOT

3.Run the CREATE PLUGGABLE DATABASE statement, and specify a local administrator for the PDB. Specify other clauses when they are required.

a.create pdbs datafile directory
mkdir -p /u02/oradata/pdbs/trsenpdb

b.runing create pdb sql on  root container.
CREATE PLUGGABLE DATABASE trsenpdb ADMIN USER trsenadmin IDENTIFIED BY oracle
  STORAGE (MAXSIZE 2G MAX_SHARED_TEMP_SIZE 100M)
  DEFAULT TABLESPACE trsen
    DATAFILE '/u02/oradata/pdbs/trsenpdb/trsen01.dbf' SIZE 50M AUTOEXTEND ON
  PATH_PREFIX = '/u02/oradata/pdbs/trsenpdb/'
  FILE_NAME_CONVERT = ('/u02/oradata/trsen/pdbseed/', '/u02/oradata/pdbs/trsenpdb/');

c.After you create the PDB, it is in mounted mode, and its status is NEW.
SYS@EMREP>select name,open_mode,status from dba_pdbs p ,v$pdbs s where p.con_id=s.con_id;
NAME                     OPEN_MODE          STATUS
---------------------------------------- -------------------- ------------------
PDB$SEED                 READ ONLY          NORMAL
PDBEMREP                 READ WRITE          NORMAL
TRSENPDB                 MOUNTED          NEW

4.Open the trsenpdb in read/write mode.

SYS@EMREP>ALTER PLUGGABLE DATABASE trsenpdb OPEN READ WRITE;
Pluggable database altered.

SYS@EMREP>select name,open_mode,status from dba_pdbs p ,v$pdbs s where p.con_id=s.con_id;
NAME                     OPEN_MODE          STATUS
---------------------------------------- -------------------- ------------------
PDB$SEED                 READ ONLY          NORMAL
PDBEMREP                 READ WRITE          NORMAL
TRSENPDB                 READ WRITE          NORMAL

二.Creating a PDB by Cloning an Existing PDB or Non-CDB
This technique clones a source PDB or non-CDB and plugs the clone into the CDB.
To use this technique, you must include a FROM clause that specifies the source.
The source is the existing PDB or non-CDB that is copied. The target PDB is the clone of the source.
The source can be a PDB in the local CDB, a PDB in a remote CDB, or a non-CDB.
When the source is a PDB in a remote CDB or non-CDB, you must specify a database link to the remote CDB or non-CDB in the FROM clause.
The CREATE PLUGGABLE DATABASE statement copies the files associated with the source to a new location and associates the files with the target PDB.

we will clone trsenpdb to clonpdb.we using the "create pluggable databaase …. from …" technique.
1.we will view all service information 

SYS@EMREP>select name,pdb from v$services;
NAME                     PDB
---------------------------------------- ------------------------------------------------------------
trsenpdb.com                 TRSENPDB
pdbemrep.com                 PDBEMREP
EMREPXDB                 CDB$ROOT
handpay.com                 CDB$ROOT
SYS$BACKGROUND                 CDB$ROOT
SYS$USERS                 CDB$ROOT

2.create datafile directory and create clonepdb pdbs. 
if the trsenpdb is remote or non-CDB,then create database link for trsenpdb. We can using the "create pluggable database xxx from xx@db_link …" statement.

mkdir -p /u02/oradata/pdbs/clonepdb

CREATE PLUGGABLE DATABASE clonepdb FROM trsenpdb
  PATH_PREFIX = '/u02/oradata/pdbs/clonepdb'
  FILE_NAME_CONVERT = ('/u02/oradata/pdbs/trsenpdb/', '/u02/oradata/pdbs/clonepdb')
  SERVICE_NAME_CONVERT = ('trsenpdb.com','clonepdb')
  NOLOGGING;

3.view the pdbs's informations;
​
SYS@EMREP>select name,open_mode,status from dba_pdbs p ,v$pdbs s where p.con_id=s.con_id;
NAME                     OPEN_MODE          STATUS
---------------------------------------- -------------------- ------------------
PDB$SEED                 READ ONLY          NORMAL
PDBEMREP                 READ WRITE          NORMAL
TRSENPDB                 READ WRITE          NORMAL
CLONEPDB                 MOUNTED          NEW

SYS@EMREP>select name,pdb from v$services;
NAME                     PDB
---------------------------------------- ------------------------------------------------------------
clonepdb.com                 CLONEPDB
trsenpdb.com                 TRSENPDB
pdbemrep.com                 PDBEMREP
EMREPXDB                 CDB$ROOT
handpay.com                 CDB$ROOT
SYS$BACKGROUND                 CDB$ROOT
SYS$USERS                 CDB$ROOT

SYS@EMREP>ALTER PLUGGABLE DATABASE clonepdb open read write;
Pluggable database altered.

SYS@EMREP>select name,open_mode,status from dba_pdbs p ,v$pdbs s where p.con_id=s.con_id;
NAME                     OPEN_MODE          STATUS
---------------------------------------- -------------------- ------------------
PDB$SEED                 READ ONLY          NORMAL
PDBEMREP                 READ WRITE          NORMAL
TRSENPDB                 READ WRITE          NORMAL
CLONEPDB                 READ WRITE          NORMAL

三.Creating a PDB by Plugging an Unplugged PDB into a CDB
This technique uses the XML metadata file that describes the PDB and the files associated with the PDB to plug it into the CDB.
The trsenpdb on emrep container , we will use XML file pluggable to CDB2.

1. to generate an XML file named trsenpdb.xml in the /u02/oradata/pdbs directory

SYS@EMREP>BEGIN
  DBMS_PDB.DESCRIBE(
    pdb_descr_file => '/u02/oradata/pdbs/trsenpdb.xml',
    pdb_name       => 'TRSENPDB');
END;
/  2    3    4    5    6  

PL/SQL procedure successfully completed.

2.check pdbs compatibility.if output is YES 

SET SERVEROUTPUT ON
DECLARE
  compatible CONSTANT VARCHAR2(3) := 
    CASE DBMS_PDB.CHECK_PLUG_COMPATIBILITY(
           pdb_descr_file => '/u02/oradata/pdbs/trsenpdb.xml',
           pdb_name       => 'TRSENPDB')
    WHEN TRUE THEN 'YES'
    ELSE 'NO'
END;
BEGIN
  DBMS_OUTPUT.PUT_LINE(compatible);
END;
/

3.close PDB trsenpdb and  then unplug it

SYS@EMREP>alter session set container=trsenpdb;
Session altered.

SYS@EMREP>shutdown immediate;
Pluggable Database closed.

SYS@EMREP>ALTER PLUGGABLE DATABASE trsenpdb UNPLUG INTO '/u02/oradata/pdbs/trsenpdb.xml';
Pluggable database altered.

4.Run the CREATE PLUGGABLE DATABASE statement, and specify the XML file in the USING clause and Open the new PDB in read/write mode.

SYS@CDB1>CREATE PLUGGABLE DATABASE trsenpdb USING '/u02/oradata/pdbs/trsenpdb.xml' NOCOPY TEMPFILE REUSE;
Pluggable database created.

SYS@CDB1>ALTER PLUGGABLE DATABASE trsenpdb open read write
Pluggable database altered.

SYS@CDB1>select name,open_mode,status from dba_pdbs p ,v$pdbs s where p.con_id=s.con_id;

NAME                     OPEN_MODE          STATUS
---------------------------------------- -------------------- ------------------
PDBTEST                  READ WRITE          NORMAL
PDB$SEED                 READ ONLY          NORMAL
TRSENPDB                 READ WRITE          NORMAL

ORACLE 12cR1 Common and Local users.

Common user is fresh concepts before 12c database .
 A common user is a database user that has the same identity in the root and in every existing and future PDB.
Every common user can connect to and perform operations within the root, and within any PDB in which it has privileges.
Every common user is either Oracle-supplied or user-created. Examples of Oracle-supplied common users are SYS and SYSTEM.

Common users have the following characteristics:
1.A common user can log in to any container (including CDB$ROOT) in which it has the CREATE SESSION privilege.but a common user
have different privilege in different pdbs.
2.The name of every user-created common user must begin with the characters c## or C##.
3.A common user resides in the root, but must be able to connect to every PDB with the same identity.
4. if c##dba is a common user that has privileges on multiple containers, 
then the c##dba schema in each of these containers may contain different objects.

Local Users in a CDB 
A local user is a database user that is not common and can operate only within a single PDB.
A local user have same funcation in a NO-CDB.

SQL> select NAME,CON_ID,PDB from v$services;
NAME                                                                     CON_ID PDB
-------------------------------------------------------------------------------------------------------------------------------- ---------- ------------------------------------------------------------
pdbemrep.com                                                                  3 PDBEMREP
EMREPXDB                                                                  1 CDB$ROOT
trsen.com                                                                  1 CDB$ROOT
SYS$BACKGROUND                                                                  1 CDB$ROOT
SYS$USERS                                                                  1 CDB$ROOT

SQL> sho con_name;
CON_NAME
------------------------------
CDB$ROOT

SQL> select username,common from dba_users

USERNAME                 COMMON
---------------------------------------- ------
ORACLE_OCM                 YES
OJVMSYS                  YES
SYSKM                     YES
XS$NULL                  YES
GSMCATUSER                 YES
MDDATA                     YES
SYSBACKUP                 YES
DIP                     YES
SYSDG                     YES
APEX_PUBLIC_USER             YES
SPATIAL_CSW_ADMIN_USR             YES
......
SQL> alter session set container=pdbemrep;
Session altered.

when we query dba_users on pdb ,then have user informations (common is NO)


SQL> show con_name
CON_NAME
--------
PDBEMREP

SQL>select username,common from dba_users
USERNAME                 COMMON
---------------------------------------- ------
BI                     NO
PM                     NO
IX                     NO
SH                     NO
OE                     NO
HR                     NO
SCOTT                     NO
ORACLE_OCM                 YES
OJVMSYS                  YES
SYSKM                     YES
XS$NULL                  YES
......

 

if we direct the pdb's any users ,then we can used network service way

first,query the listener informations and reference the servcie informations from v$services


LSNRCTL for Linux: Version 12.1.0.2.0 - Production on 04-NOV-2016 19:40:43
Copyright (c) 1991, 2014, Oracle.  All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=sitgc.trsen.cn)(PORT=1521)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 12.1.0.2.0 - Production
Start Date                03-NOV-2016 15:49:10
Uptime                    1 days 3 hr. 51 min. 35 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u01/app/oracle/product/12.0.1.2/db_1/network/admin/listener.ora
Listener Log File         /u01/app/oracle/diag/tnslsnr/sitgc/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=sitgc.trsen.cn)(PORT=1521)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=sitgc.trsen.cn)(PORT=5500))(Security=(my_wallet_directory=/u01/app/oracle/admin/trsen/xdb_wallet))(Presentation=HTTP)(Session=RAW))
Services Summary...
Service "EMREPXDB.com" has 1 instance(s).
  Instance "EMREP", status READY, has 1 handler(s) for this service...
Service "trsen.com" has 1 instance(s).
  Instance "EMREP", status READY, has 1 handler(s) for this service...
Service "pdbemrep.com" has 1 instance(s).
  Instance "EMREP", status READY, has 1 handler(s) for this service...
The command completed successfully


[oracle@sitgc:/tmp]$sqlplus sh/sh@sitgc.trsen.cn:1521/pdbemrep.com
SQL*Plus: Release 12.1.0.2.0 Production on Fri Nov 4 19:40:54 2016
Copyright (c) 1982, 2014, Oracle.  All rights reserved.
Last Successful login time: Fri Nov 04 2016 18:44:25 +08:00
Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
SQL> show user;
USER is "SH"
SQL> show con_name;
CON_NAME
------------------------------
PDBEMREP
SQL> select tname from tab;
TNAME
--------------------------------------------------------------------------------
SALES
COSTS
TIMES
PRODUCTS
CHANNELS
PROMOTIONS