Enabling Search Framework Real Time Indexing (RTI)

As we prepare our HCM PUM 46 training servers, we decided to enable Real Time Indexing (RTI). We found several great sample videos and resources but still made a few mistakes. We are sharing our adventure to help you as you implement RTI.

Before implementing RTI, we have a couple prerequisites. First, we must have a working Elasticsearch (OpenSearch) service. Second, we should have some indexes deployed. Since our training focus is PeopleTools, we usually deploy the PTSF_RECENTMENU and the PTPORTALREGISTRY indexes. To summarize the prerequisites, we need a:

• Working Elasticsearch (OpenSearch) installation and
• Deployed indexes

Elasticsearch (OpenSearch) does not store its data in the PeopleSoft database. Instead, we move data into the Elasticsearch (OpenSearch) index storage through scheduled batch processes. The point of RTI is to reduce latency by eliminating scheduling; get data into indexes as fast as possible: Enter data → Search for data.

RTI works by using the following:

• Triggers to insert rows into a staging table and
• A Process Scheduler server process that checks for new rows in the staging table

Enabling RTI is a multi-step process requiring:

• Process Scheduler-specific configurations and
• Index-specific configurations

Process Scheduler Configuration

Let's start with the Process Scheduler Server. Using the psadmin command, make sure Real Time Indexing is enabled, as demonstrated in the following screenshot:

This turns on the "watcher" process. On interval, the PSRTISRV service polls the RTI staging table, PS_PTSF_RTISTG, for new rows to process. We can tune the number of RTI instances through the psprcs.cfg file.

[PSRTISRV]
;=========================================================================
; Settings for Real Time Indexing watchdog
;=========================================================================
;-------------------------------------------------------------------------
Min Instances=1
Max Instances=1

We may configure the polling interval through the Search Instance Search Options Config (PeopleTools > Search Framework > Administration > Define Search Instances). The last several options relate to RTI:

The Heartbeat Interval determines how often the RTI process checks for new rows in the RTI staging table (PS_PTSF_RTISTG). The Real Time Indexing Set Size specifies how many rows to process on each interval. The point is to tune these two numbers for optimal performance. The shorter the interval, the closer to real time, but with a performance cost. The longer the interval, the less the performance impact. Too long of an interval, however, and the number of transactions per interval may exceed the Set Size threshold, meaning RTI may only clear the queue after hours, creating a larger backlog with each run.

As a side note, PeopleSoft queues transactions for RTI through database triggers. A bulk update to a triggered table may generate more transactions than RTI can clear in a single polling interval, creating a latency effect for online transactions. Given the default configuration parameters, PeopleSoft will index 300 transactions every 2 seconds. If we batch update 30,000 rows of data, we may have to wait 4 minutes before that data clears the queue. 300,000 rows of data? Roughly 35 minutes. With this in mind, we may want to remove an index from RTI before a batch update, run the update, re-enable, and then schedule an incremental build to catch up. That way, intermittent, unusual batch processing doesn't interfere with online RTI processing.

Index Configuration

After configuring server settings, we choose which indexes to enable. Navigate to PeopleTools > Search Framework > Configure Realtime Indexing to select an index. To start with, pick indexes that are Disabled. These are indexes that contain RTI metadata, making them trivial to enable. Indexes marked as Not Configured may be enabled but will require additional configuration before enabling.

In our demo environments, we enabled PTSF_RECENTMENU. To enable an index:

• Check the box Enable Real Time Indexing,
• Save, and
• Click the Trigger SQL link to download database-specific triggers.

When you click the Trigger SQL link, PeopleSoft will either open three new tabs or download three new files (it depends on browser configuration and registered handler for SQL files). The file name identifies the database: ora for Oracle, mss for Microsoft SQL Server, and db2 for DB2. If your browser opens them in new tabs instead of downloads, scroll to the end of the address bar to confirm the file name. This is where I made a mistake. I had so many open tabs that I didn't notice PeopleSoft opened three new ones. I only saw one and kept trying to run the DB2 script against my Oracle database.

Alternatively, as described by Ravi and Ramasimha in their Quest Recorded Webcast, we can let App Designer build the proper database triggers for us. I appreciate the App Designer integration because it ensures future record changes won't delete the RTI trigger scripts. It also means I don't have to log into a separate SQL tool or verify that I chose the correct database script.

After enabling RTI for an index, PeopleSoft recommends running a full or incremental sync to ensure all data exists in the target index. After that, you can cancel any scheduled run controls for that index, as RTI will now manage the incremental updates for that specific index.

Here are some great resources with additional information about RTI:

• Siva Koya's RTI YouTube Video
• PeopleSoft Tutorial
• Quest Recorded Webcast

Is RTI Really "Real time"?

Realtime usually means data automatically flows from one system to the next. Create data → Send data. If RTI were truly real time, each update would send immediately. But if you have up to 300 updates within a 2-second period, a real time interface would create 300 connections to incrementally send 300 updates. PeopleSoft's near real time approach, however, is a lot like App Engine's Bulk Update. Another way to think of it is that RTI is a shuttle with room for 300 passengers. A shuttle arrives once every 2 seconds to pick up however many passengers are waiting. This is far more efficient than sending up to 300 vehicles individually. And even better, through parameters, we can tune the interval and passenger count to achieve the best experience for our users.

Which Indexes Should You Enable?

We recommend starting with PTSF_RECENTMENU. This allows the portal search feature to suggest user and enterprise recent items within search results. Next, we suggest enabling indexes used by component keyword search so your users may view transactions immediately after creation.

At JSMpros, we teach PeopleTools concepts every week. Each student who attends a live training class receives a properly sized, personal training server with relevant PeopleTools and application versions for the duration of training. This is one of the ways we provide you with an unparalleled training experience! Check out our website to see what we are offering next!

Access all of our content anytime from anywhere through our all-access subscriptions. Check it out!