Useful IBM FileNet Technote when setting up FileNet v5.2 environment

Below are some of the technotes that I found useful in resolving known issues while setting up FileNet v5.2 environment. 

Also has some other technotes which are good to know when setting up FileNet v5.2 environment.

• Process service applets cannot be launched in Workplace due to java.lang.NoClassDefFoundError exceptions after applying Content Platform Engine (CPE) 5.2.0.3

http://www-01.ibm.com/support/docview.wss?uid=swg21669539

• Process service applets cannot be launched in Workplace XT due to java.lang.NoClassDefFoundError exceptions after applying Content Platform Engine (CPE) 5.2.0.3

http://www-01.ibm.com/support/docview.wss?uid=swg21668501

• Configuration Manager CMUI displays blank boxes instead of characters in the graphical interface

http://www-01.ibm.com/support/docview.wss?uid=swg21663471

• Configuring the Rendition Engine Connection when using an Oracle RAC database

http://www-01.ibm.com/support/docview.wss?uid=swg21670179

• How to add system properties and FileNet.properties properties

http://www-01.ibm.com/support/docview.wss?uid=swg21496579

• Content Collector does not detect cluster or other nodes in version 4.0

http://www-01.ibm.com/support/docview.wss?uid=swg21675812

• Troubleshooting problems with viewers in IBM Content Navigator

http://www-01.ibm.com/support/docview.wss?uid=swg27039281

• New install of IBM FileNet P8 Rendition Engine (RE) cannot publish to .PDF

http://www-01.ibm.com/support/docview.wss?uid=swg21389813

• While attempting to configure the JPSMonConfJMXMon Monitor, an UnknownHost error was reported for the event in IBM Enterprise Content Management System Monitor

http://www-01.ibm.com/support/docview.wss?uid=swg21625580

• IBM Enterprise Records declaration error when using IBM FileNet Content Manager V5.2 FP3

http://www-01.ibm.com/support/docview.wss?uid=swg21669418

• Running P8 Client Applications with WebSphere Application Server 8.5.x

http://www-01.ibm.com/support/docview.wss?uid=swg21655351

• Running P8 Client Applications with WebSphere Application Server 8.0.0.x

http://www-01.ibm.com/support/docview.wss?uid=swg27023814

• IBM Enterprise Records Web application gives a "Null" error when attempting to login

http://www-01.ibm.com/support/docview.wss?uid=swg21626250

• Manually creating a configuration profile when you cannot connect to the web application server

http://www-01.ibm.com/support/docview.wss?uid=swg27039360

• Configuring single sign-on for Workplace XT by using Kerberos/SPNEGO

http://www-01.ibm.com/support/docview.wss?uid=swg27019844

• Configuring single sign-on for IBM Content Navigator by using SPNEGO/Kerberos on WebSphere Application Server (IBM FileNet P8)

http://www-01.ibm.com/support/docview.wss?uid=swg27036837

• How to disable Content Engine (CE) client-side Buzz (com.filenet.wcm.api) logging

http://www-01.ibm.com/support/docview.wss?uid=swg21683849

• Who's using my port? (AIX, Linux, Windows example)

http://www-01.ibm.com/support/docview.wss?uid=swg21264632

• Data codes related to 'LDAP: error code 49' with Microsoft Active Directory

http://www-01.ibm.com/support/docview.wss?uid=swg21290631

• Permission requirements in a Microsoft Exchange 2007 or Microsoft Exchange 2010 environment

http://www-01.ibm.com/support/docview.wss?uid=swg21431315

• IBM Content Navigator known limitation: Enabling logs for viewer server side operations

http://www-01.ibm.com/support/docview.wss?uid=swg21600364

Converting FileNet ID's based on Oracle/DB2 from byte reversal format to GUID [and vice-versa]

If you have been working on FileNet based on various database, you would be familiar that MSSQL stores the FileNet GUID is the same exact format as what can be seen in FileNet Enterprise Manager using the datatype 'uniqueidentifier'

But when using DB2 or Oracle, the data related to GUID is stored in a byte reversal format. If you are not really familiar with it, there is a technote available from IBM explaining this and the process to get the GUID out of the byte reversed format.

http://www-01.ibm.com/support/docview.wss?uid=swg21587420

I have attached a small html file which can perform this operation in bulk, for use during testing or for any other reason where you might find it useful. It can also can perform GUID to hexadecimal format. 

• Download the html file locally on your machine, open it in your favorite browser. 

GUIDv3.zip

• Copy the byte reversed string from the database

• Paste the byte reversed string into the text area, one per line.

• Click the 'Get GUID' button to get the GUID format of the byte reversed string.

Use the GUID to search within FEM, ACCE or using FileNet API's.

You can also used the GUID string from FEM/ACCE or any other source and convert them to a hexadecimal byte reversed format using the same utility for use in SQL script. You can also have a mix of these values for conversion.

Setting up Tivoli Directory Server - Quick and Easy

Whenever you are setting up a FileNet System you would require a Directory Server as one of the pre-requisite. Normally every large enterprise would already have Directory Server and you just need to have FileNet hooked up to this Server.

But when you are setting up a demo or implementing a solution at a small/medium size customer who are beginning the process of setting up the system, you would end up requiring to install the Directory Server.

IBM Tivoli Directory Server comes bundled with FileNet for using it as a Directory Server. Normally IT staff find it intimidating to setup the Tivoli Directory Server (TDS) just because they have never tried it before and tend to use Active Directory Server in most cases.

Setting up of TDS is quite simple and you should be up and running within 30-40 minutes (atleast for a simple setup). The below procedure shows how to setup TDS and also import users and groups into TDS using a ldif file.

The below procedure is for TDS v6.3 on Linux, but the steps are the same for Windows or AIX and for earlier 6.x version of TDS.

NOTE: There are restriction to the usage of the bundled TDS software, refer to the license document for more details.

Install IBM TDS

Using root, untar the installer file into a temp location

Change directory to tds folder, run the install_tds.bin (In case of Windows, just double click the install_tds.exe file)

 Select the language and click on OK

 

Click Next to start the installation

Read the license and click on I accept if you agree. Click on the Next button to continue with the installation

 If an existing DB2 exists, it should get automatically detected, else a new one would be installed by the installer, click Next

Choose Typical installation and click Next

Click on Install to start the installation

 When prompted for password, enter a suitable password, for the Encryption seed enter a 12 character string, click Next

 Click Finish to complete the installation

TDS Instance Administration Tool should get automatically started up

Create a suffix for the ldap users and groups

Click on the Manage button on the TDS Instance Administration Tool

Select the Manage suffixes and add an entry into Suffix DN, Eg. dc=com and click Add

Click OK and close the window.

Import ldif file to create user an groups

Create a .ldif file as shown below

Sample is shown for a user (aeadmin) and group (aeadmins) creation. The password for the user is 'password' as shown below, change it to a suitable value for your environment.

dn: o=ibm,dc=com

objectclass: organization

objectclass: top

o: ibm

dn: o=ecm,o=ibm,dc=com

objectclass: organization

objectclass: top

o: ecm

dn: cn=aeadmin,o=ecm,o=ibm,dc=com

userpassword: password

objectclass: inetOrgPerson

objectclass: organizationalPerson

objectclass: person

objectclass: top

sn: aeadmin

cn: aeadmin

dn: cn=aeadmins,o=ecm,o=ibm,dc=com

objectclass: groupOfNames

objectclass: top

cn: aeadmins

member: cn=aeadmin,o=ecm,o=ibm,dc=com

Click on the Manage button on the TDS Instance Administration Tool

Select the LDIF tasks and Import LDIF data, choose the ldif file using the Browse button and then click Import.

The import should start and the message would be displayed regarding the import operation.

Start the TDS Server

On the TDS Instance Administration Tool click on the Start/Stop button and click on the Start Server button.

Once started a prompt is shown, click OK

Click Close button on the TDS Instance Administration Tool

Now the TDS service is up and running for use with FileNet or any other system.

Extending IBM CM Database View to show current Process Step Name

In CM8, whenever a new Item Type is created, it creates a number of tables in the Library Server database in order to support the Item Type classification selected for the Item Type.

The tables created have dynamic name assigned to them, so it is not possible to preempt the names. However for every Item Type there is a view generated in the database automatically, with the same name as the Item Type with 001 suffixed, exceptions to these are when the Item Type name is bigger than 16 chars (I guess, do not recollect it that well). This view provides columns for all the attributes defined for the Item Type.

These views are of great help when generating reports as you get to work at the DB layer and query the required data knowing that the performance would be the best without any additional overhead that API's have. 

NOTE: It is advised to use these views for read-only operation, no updates should be done directly at the DB layer without fully understanding the impact that it might have on the stability of the system.

However such views do not have any information related to the process (Document Routing). Many a times when generating the reports you would require to know what is the current step at which the work item is at for a given Item of an Item Type. By following the below procedure you would be able to expose the current workflow step for all the Items in an Item Type view.

The following example uses the NOINDEX Item Type, but same procedure applies for other Item Type as well.

Procedure

• Connect to the Library Server database
• Get the (existing) view definition for the Item Type (NOINDEX in this example)

db2 select text from syscat.views where viewname='NOINDEX001'

-----------------------------------------------------------------------------

CREATE VIEW ICMADMIN.NOINDEX001 (COMPCLUSTERID, COMPONENTID, ITEMID, VERSIONID, ACLCODE, SEMANTICTYPE, EXPIRATIONDATE, COMPKEY, CREATETS, CREATEUSERID, LASTCHANGEDTS, LASTCHANGEDUSERID, SOURCE, USER_ID, TIMESTAMP, ICMMANAGEDRECORD) AS SELECT COMPCLUSTERID, COMPONENTID, ITEMID, VERSIONID, ACLCODE, SEMANTICTYPE, EXPIRATIONDATE, COMPKEY, CREATETS, CREATEUSERID, LASTCHANGEDTS, LASTCHANGEDUSERID, ATTR0000001000, ATTR0000001001, ATTR0000001002, ICMMANAGEDRECORD FROM ICMADMIN.ICMUT01000001 WHERE   (EXISTS ( SELECT 1  FROM ICMADMIN.ICMSTCOMPILEDACL ICMC, ICMADMIN.ICMSTUSERS U WHERE (ICMC.ACL=1 OR ICMC.ACL=-1) AND ((U.USERID=USER AND ICMC.UNUM=U.UNUM) ) AND ICMC.RPRIV='1') )

• The above view need to be rewritten to include 2 additional tables, one for the work nodes (ICMUT00202001) and the other for the work packages (ICMUT00204001) table. Give a new name to the view, an alias to the Item type table as 'IT' and prefix attributes columns with the 'IT' prefix. The rewritten query would be as below

CREATE VIEW ICMADMIN.NOINDEXWF001 (COMPCLUSTERID, COMPONENTID, ITEMID, VERSIONID, ACLCODE, SEMANTICTYPE, EXPIRATIONDATE, COMPKEY, CREATETS, CREATEUSERID, LASTCHANGEDTS, LASTCHANGEDUSERID, SOURCE, USER_ID, TIMESTAMP, ICMMANAGEDRECORD, STEP_NAME) AS SELECT IT.COMPCLUSTERID, IT.COMPONENTID, IT.ITEMID, IT.VERSIONID, IT.ACLCODE, IT.SEMANTICTYPE, IT.EXPIRATIONDATE, IT.COMPKEY, IT.CREATETS, IT.CREATEUSERID, IT.LASTCHANGEDTS, IT.LASTCHANGEDUSERID, IT.ATTR0000001000, IT.ATTR0000001001, IT.ATTR0000001002, IT.ICMMANAGEDRECORD, WN.WORKNODENAME FROM ICMADMIN.ICMUT01000001 AS IT, ICMADMIN.ICMUT00202001 AS WN, ICMADMIN.ICMUT00204001 AS WP WHERE (WP.RTARGETITEMID = IT.ITEMID AND WP.ITEMID = WN.ITEMID) AND  (EXISTS ( SELECT 1  FROM ICMADMIN.ICMSTCOMPILEDACL ICMC, ICMADMIN.ICMSTUSERS U WHERE (ICMC.ACL=1 OR ICMC.ACL=-1) AND ((U.USERID=USER AND ICMC.UNUM=U.UNUM) ) AND ICMC.RPRIV='1') )

The above query has a join on the Item Type table and the Work package table for getting documents with processes and another join on the Work package and Work node table for getting the step name.

The procedure is now complete. 

The output of a query using the existing view would be as below

db2 "SELECT ITEMID FROM NOINDEX001"

ITEMID

--------------------------

A1001001A07C26C10637B19967

A1001001A13F29A61324A04886

A1001001A13F29C05550E23937

The output using the newly created view with the STEP_NAME would be as below, only Items having a active workflow is shown

db2 "SELECT ITEMID, STEP_NAME FROM NOINDEXWF001"

ITEMID                     STEP_NAME

-------------------------- --------------------------------

A1001001A13F29C05550E23937 INDEX

Optimizing DB2 query performance for FileNet and IBM CM

Several times the tuning done using the FileNet Enterprise Manager (FEM) or IBM CM Administration Client like creating indexes for properties and attributes do not really improve the performance of a query.

In such a case the DB2 Advisor tool can really help and improve the query performance several times over. 

I had faced issues before with IBM CM, where the eClient application was taking upto 30 seconds or more to open up a page for the Worklist and after tuning the indexes for the query the same page was loading under 2 second. Same applies with FileNet query related indexes.

Before we start using the DB2 Advisor tool we need to collect all the queries which are taking a lot of time for processing. The SQL query can be obtained for both FileNet and IBM CM by enabling trace level logging. The method for enabling trace level logging in both the products is as given below.

FileNet DB Trace Level Logging

Login to FEM, right click on your Domain and select Properties
Click on the Trace Control tab, click on the Enable Trace Logging button
Check the Database check box as shown below

IBM Content Manager Travel Level Logging

Login to IBM Content Manager Administration Client
Click Tools in the menu bar, select Log Configuration
Click on the Library Server button, select the Logging level as Trace (full)
NOTE: We do not need the Resource Manager trace logging for RMDB database queries as they do not really take up much processing time.

It is highly suggested that this trace level logging be enabled only during off business hour as this will affect the performance of the system and also would write a lot of log which can get overwhelming when trying to pin point the query that is creating the performance issue.

Reproduce the same condition for which you were seeing a slow query performance so that the SQL query gets logged into the trace log file.

Collect all the queries from the trace file which are taking a lot of time processing in a single plain text document with each line consisting of one query. Once you have the list, you can use the DB2 Advisor tool to get index recommendation.

Enable the DB2 Advisor if it is not already enabled by running the below command in db2cmd command prompt as shown using a DB2 administrator account.

db2 -tvf <db2-install-location>\MISC\EXPLAIN.DDL

To use the DB2 advisor, you need to be logged in as a DB2 administrator and issue the below command using the db2cmd command prompt.

db2advis -d <database-name> -i <plain-text-document-with-the-sql-query> -t 5 -a <schema-username> -x <schema-password>

Replace the <database-name> with your db name
            <plain-text-document-with-the-sql-query> with your plain text file                containing all the queries
            <schema-username> with your schema username
            <schema-password> with your schema password

This should throw up a big list of Recommended, Used Existing and Unused Existing Indexes. It is a good idea to redirect the output of the command to a text file so that you can view the entire list using the text editor as below.

Copy the recommended indexes without into a new plain text file and then implement these indexes by running the following command on db2cmd command prompt

db2 -tvf <file-name> 

Alternatively if the recommended index list is small you could just copy each recommended indexes and run them individually on the db2cmd prompt, 'db2' needs to be prefixed for each of the command as shown below. 

A database connection is required using a db2 admin account in either case to create the indexes.

You can repeat the process of getting the db2advis recommendation again after implementing the previous recommendation to further tune the queries. It is however not advised to iterate more than 3 times as this would create additional overhead for addition of record and also use additional disk space for maintaining the indexes.

Most importantly do not forget to disable the logging once the exercise is completed.

Enabling ICC to connect to Exchange Server 2007/2010 using Outlook Client 2003

There was a requirement for me to setup a demo machine using ICC v2.2.2 for archiving emails from Exchange 2010 server. The ICC was hosted on a Windows 2003 server with Outlook 2003 configured for accessing the mailboxes from Exchange Server 2010

Outlook 2003 supports connecting to Exchange 2010 server for mail access, but ICC does not work with the MAPI API shipped with Outlook 2003 with Exchange Server 2010. If such an ICC task is run, you would be able to see MAPI_E_NETWORK_ERROR being logged into the email connector log file of ICC. IBM Support also confirmed that the Outlook client on the ICC server needs to be 2010 version for Exchange 2010 server.

The only way to get ICC to work with Exchange Server 2010 was to install the Outlook 2010 client on the ICC server machine, but the Outlook 2010 is not supported on Windows Server 2003 Operating System (on which the ICC was setup).

For this problem I found a workaround using the Microsoft Exchange Server MAPI Client and Collaboration Data Objects 1.2.1 package. The below mentioned procedure details the same.

NOTE: 

Before you follow this procedure, ensure that you can access the Exchange 2010 server mailboxes using the Outlook 2003 client on the ICC server.

After following the below mentioned procedure, the outlook client on the ICC server will not be usable, but will work for ICC.

Procedure

1. Download the Microsoft Exchange Server MAPI Client and Collaboration Data Objects 1.2.1 from Microsoft site using the link http://www.microsoft.com/en-us/download/details.aspx?id=1004
2. The downloaded package cannot be installed on a machine which already has Outlook, so install this on any other machine which does not have Outlook.
3. Once installed on another machine, the binary files from the installer are laid down at the location C:\Program Files (x86)\ExchangeMapi, copy all the (11 in total) file into a temp directory on the ICC server
   
4. On the ICC server, exit from the outlook client if it is opened, stop ICC task routing service along with the email source connector.
5. Go to the Outlook MAPI folder location on the ICC server and take a backup. The location for this would be C:\Program Files (x86)\Common Files\System\MSMAPI\1033
6. Copy the files from temp folder from Step 3 into the folder mentioned in step 5, choose to replace all the existing files with the newly copied one. Ensure that all the files are replaced and does not fail while replacing.
7. Start the ICC task route and email source connector.

Now the emails should get archived successfully using ICC.

This procedure should work with ICC V3.0 and V4.0 as well and also with Exchenage 2007 Server, but I have not tested it. 

Also I would not suggest this approach for setting up any ICC Production environment.

Selectively restoring documents from database backup for IBM Content Manager

The below procedure was followed for one of the customers where an utility had unintentionally deleted off records from production system for documents which were older than a set number of years.

The customer had the backup of the Library Server (LS), Resource Manager (RM) and LBOSDATA directory, but restoring them directly would have wiped out the data which were added after the backup date.

Recreating the document again using the API would affect a lot of business logic as new Event Log entries would have been created and the system attributes like create date, modified date, etc. would also change.

Document deletion in IBM Content Manager

The deletion of documents in IBM Content Manager causes records to be immediately deleted from LS database and an entry is made into the ICMSTITEMSTODELETE table for later  deletion from RM Database and LBOSDATA directory. The RMMigrator and RMPurger then use this information to delete the records from RM Database and LBOSDATA directory whenever their scheduled cycle starts.

Depending upon whether the RMMigrator and RMPurger service are started or stopped, the data restoration for Resource Manager Table and LBOSDATA may or may not be required in your case.

In our case, the customer had not started the RMMigrator and RMPurger service, so only the Library Server database records had to be restored and ICMSTITEMSTODELETE table needed to be cleared, while the RM table and LBOSDATA were intact. 

In the below mentioned procedure, I have provided details for restoring data into LS, RM database and LBOSDATA. Please skip the RM database and LBOSDATA restoration steps if they have not been affected.

Procedure

NOTE: Depending on the features used in your Item Type, such as Reference Attribute, Child Components, etc. additional tables are going to be affected and the procedure for recovering from those is not provided here as part of the document. Here the default NOINDEX item type is used for the example. The procedure should work on IBM CM v8.3.x version and higher. If you need professional support contact your local IBM Lab Services.

Step 0: 

Restore the databases (LS and RM) on a different instance of DB2 (Generally the existing default instance would be db2admin on Windows platform / db2inst1 on unix platform).

Stop all client applications connecting to IBM Content Manager. 

Stop Resource Manager Application and its utilities (RMMigrator, Purger, etc.).

Take a fresh backup of the existing LS, RM database and LBOSDATA directory to ensure fallback in cause of issues with the procedure.

Follow the steps from 1 to 4 on the newly restored database and steps 5 to 8 on the existing database.

Step 1: 

For a given item type name add a suffix of 001 to get the respective view name, Ex. for the NOINDEX Item Type, the view name would be NOINDEX001

Use the following query to get the underlying tables Item Type ID

db2 SELECT COMPONENTVIEWNAME, ITEMTYPEID FROM ICMSTCOMPVIEWDEFS WHERE COMPONENTVIEWNAME='NOINDEX001'

COMPONENTVIEWNAME  ITEMTYPEID

------------------ -----------

NOINDEX1001               1007

Use the Item Type ID data from above output in the query below

db2 select COMPONENTVIEWNAME, ComponentTypeID from ICMSTCompViewDefs where

itemtypeid=1007

COMPONENTVIEWNAME  COMPONENTTYPEID

------------------ ---------------

NOINDEX1001                   1014

ICMParts1007001               1015

The underlying data tables for the Item Type are in the form ICMUT0XXXX001, so from the above result it would be ICMUT01014001 and ICMUT01015001

Apart from these tables, there are other standard tables such as ICMSTRI001001 (Reference Integrity Table), ICMSTITEMS001001 (Item Versions Table) and ICMUT00300001 (Base Table) from where the data for a document has to be extracted.

More details on these tables are available on ‘IBM Content Manager Info Center > Reference > System control tables’ section.(http://pic.dhe.ibm.com/infocenter/cmgmt/v8r4m0/topic/com.ibm.systemtables.doc/systemcontroltables.htm)

Step 2: 

Using the ICMUT01015001, for a given itemid get the rtargetitemid using the query below, for example itemid='A1001001A13E13A82739B09289'

db2 select rtargetitemid from ICMUT01015001 where itemid='A1001001A13E13A82739B09289'

RTARGETITEMID

--------------------------

A1001001A13E13A82739B17487

Step 3: 

Using now both the itemid and rtargetitemid, extract data from the following LS tables (replace the <itemid>, <rtargetitemid> and <filename> place holders below). Give distinct <filename> in the below command.

db2 export to <filename>.ixf of ixf select * from ICMADMIN.ICMUT01014001 where itemid=<itemid>

db2 export to <filename>.ixf of ixf select * from ICMADMIN.ICMUT01015001 where itemid=<itemid>

db2 export to <filename>.ixf of ixf select * from ICMADMIN.ICMSTRI001001 where sourceitemid=<itemid>

db2 export to <filename>.ixf of ixf select * from ICMADMIN. ICMSTITEMS001001 where itemid=<itemid>

db2 export to <filename>.ixf of ixf select * from ICMADMIN. ICMSTITEMS001001 where itemid=<rtargetitemid>

db2 export to <filename>.ixf of ixf select * from ICMADMIN.ICMUT00300001 where itemid=<rtargetitemid>

Step 4: 

Using the rtargetitemid from above, extract data from the RM table

db2 export to <filename>.ixf of ixf select * from RMADMIN. RMOBJECTS where obj_itemid=<rtargetitemid>

Step 5:

Import the data that was exported from LS tables from the above steps

db2 import from <filename>.ixf of ixf messages <filename>.msg insert into ICMADMIN.ICMUT01014001

db2 import from <filename>.ixf of ixf messages <filename>.msg insert into ICMADMIN. ICMUT01015001

db2 import from <filename>.ixf of ixf messages <filename>.msg insert into ICMADMIN. ICMSTRI001001

db2 import from <filename>.ixf of ixf messages <filename>.msg insert into ICMADMIN.ICMSTITEMS001001

db2 import from <filename>.ixf of ixf messages <filename>.msg insert into ICMADMIN.ICMSTITEMS001001

db2 import from <filename>.ixf of ixf messages <filename>.msg insert into ICMADMIN. ICMUT00300001  

Ensure that the restored itemid entries are not present in ICMSTITEMSTODELETE table and if it is present then delete it

db2 delete from ICMADMIN.ICMSTITEMSTODELETE where itemid=<itemid>

Step 6: 

Import the data that was exported from RM tables from the above steps

db2 import from <filename>.ixf of ixf messages <filename>.msg insert into RMADMIN. RMOBJECTS  

Step 7: 

For the LBOSDATA directory, the documents can be directly merged from backup, with conflicting documents not to be overwritten.

Step 8:

Start the resource Manager application, search for the newly restored data using IBM CM Client for Windows or eClient application and then retrieve the document. If every part of the document data has been successfully restored you should be able to search and view the document along with its content.

This entire procedure can be automated by writing a small program with the above logic.