Oracle 11g RAC database on ASM, ACFS or OCFS2
I see a lot of questions on shared file systems that can be used when people move from single instance to Oracle RAC database and Grid Infrastructure. The most common question is the following: “Should I place archive logs in OCFS2 or ASM or ACFS of 11g?” I’ll try to clarify this topic below.
First we need to understand a separation between Oracle database files and non-database files.
Database files belong to an Oracle database and include control files, data files, archived logs, online redo logs, etc. Oracle ASM is the preferred storage manager for all database files. It has been specifically designed and optimized to provide the best performance for database file types. The file list supported by Oracle ASM becomes longer with every new version and it includes nowadays also spfile, RMAN backupsets and some other file types. You can however use other shared cluster file systems, including OCFS2 to store Oracle RAC database files.
Non-database files are everything that does not belong to database files including Oracle binaries (ORACLE_HOME, etc.). Oracle ACFS is the preferred file manager for non-database files. It is optimized for general purpose files and can be shared across the RAC cluster. You can use both ACFS or local file system, like Ext3 to store Oracle database binaries (ORACLE_HOME).
ACFS is a part of Oracle Grid Infrastructure. It cannot be used for a root file system, boot file system, or a file system containing the executables for Grid Infrastructure. It is also not supported to store in ACFS any files that can be natively stored in ASM; i.e., it is not supported to store Oracle database files (controlfiles, datafiles, archived logs, online redo logs, etc.). Also, the Grid Infrastructure Home cannot be installed in ACFS, it must be installed in a separate file system; e.g.; ext3. You can use ACFS though for Oracle binaries (ORACLE_HOME), but I would not recommend this.
However, and this is new, since Oracle database version 11.2.0.3 archive logs are supported in ACFS. See details: http://t.co/vOT8bMrE. I reflected everything in below support matrix.
Why not OCFS2
You can use OCFS2 to store both database file and Oracle database binaries (ORACLE_HOME) However Oracle does not recommend using OCFS2 anymore for RAC systems, for the following reasons:
– OCFS2 and Grid infrastructure (Oracle Cluster-ware) are two separate cluster-wares. Therefore you would be running with 2 cluster-wares instead of one.
– OCFS2 will not gain any new functionality in regard of databases. OCFS2 will be used for Oracle VM and all new functionalities are more targeted in this direction than for databases.
As a summary, the recommendation of Oracle is clearly to use ASM for database files, and for everything which cannot be placed directly in ASM to use ACFS. However OCFS2 will be continued to be supported (at least at the moment).
Below is a support matrix for Oracle RAC / Grid Infrastructure installation
Supported Storage Option | Voting/OCR | GRID Software | Database files | Database Software |
Automatic Storage Manager (ASM) | Yes | No | Yes | No |
ASM Cluster File System (ACFS) | No | No | Partially (11.2.0.3+) | Yes |
Oracle Cluster File System (OCFS2) | Yes (1.4.1+) | Yes (1.4.1+) | Yes (1.4.1+) | Yes (1.4.1+) |
Red Hat Global File System (GFS) | No (11g+) | No (11g+) | No (11g+) | No (11g+) |
NFS (certified only) | Yes | Yes | Yes | Yes |
Local Storage | No | Yes | No | Yes |
.
Check out RAC Technologies Matrix for Linux Platforms
thanks Kirill for you informations, keep on it!
Thanks Kirill, nice post.
if ACFS is recommended for non oracle non datafiles such as binary and why so many shops use ASM for oracle datafiles including ocr and voting file but use flat file(regular file system) for oracle binary files ?
>> why so many shops use ASM for oracle datafiles including ocr and voting file but use flat file(regular file system) for oracle binary files ?
If we are talking about RAC / RAC1Node DB setup then having oracle datafiles including ocr and voting file on shared file location is a must.
So what else you can use for no extra costs than oracle ASM?? probably nothing.
Regarding Oracle database binaries you can install them locally on regular file system of every cluster node. This setup is preferable since:
you have a possibility of RAC rolling patching (planned downtime can be reducing) and native file system is more stable than ACFS.
Yes, you can have shared binary set on ACFS but rolling patching is not available in this case. Furthermore, in case your ASM fails, all your ACFS file systems will be unavailable including Oracle ADR with all the DB logs and traces, etc.
This is my recommendation
— Kirill
Hi Kirill,
Nice post !! It would be very helpful if you can publish the metrics about which storage system was used for which layer as per Oracle version. Example RAW devices were used for OCR and Voting Disk until 11.1 and they are not supported anymore.
If we can have that info at 1 place, it will avoid lot of confusion for everyone.
Thanks,
Advait Deo