Showing posts with label AppEngine. Show all posts
Showing posts with label AppEngine. Show all posts

Wednesday, June 12, 2024

Is "Restartable" an App Engine Best Practice?

The first thing I do when creating an App Engine is disable restart. Because restartable is the default, it might seem like Restartable is a best practice. But is it?

A restartable App Engine is a batch program that iterates over data and commits changes on an interval. This behavior contrasts with the default behavior, which is to roll back changes on failure. A restartable program restarts at the last checkpoint. Without restart, ALL data must be reprocessed on failure. The obvious benefit is that restartable App Engines don't have to reprocess successful data. This sounds fantastic! But is it?

There are two main data processing strategies:

  • Set-based processing
  • Row-by-row processing

Set-based processing allows the database to optimize and execute commands at the database. Row-by-row processing, however, requires each row of data to be transferred to the process scheduler server and then processed. Row-by-row processing, therefore, is dramatically slower.

App Engines utilizing either strategy may be restartable, but a restartable set-based program would be rare. Restart is only valuable on failure. Before restarting, we would fix whatever data caused an error so we may reprocess that data. With set-based processing, that error would already be stored in an intermediate downstream processing table. Therefore, row-by-row processing is the only processing method that seems relevant for restart.

Another problem with a Restartable App Engine is that it must commit on intervals to benefit from the restart. What makes this a problem is that databases, such as Oracle, close any open cursors on commits. Therefore, a Restartable App Engine must also re-fetch (close and reopen) SQL cursors on every commit.

What does this mean? A Restartable App Engine may be the slowest, least-performing option available! It is an option we may use when it makes sense, but should it be the default? What do you think?

Ready to take your PeopleTools skills to the next level? Become a subscriber to gain access to our vast library of PeopleTools training courses, hands-on activities, downloads, instruction videos, and more!

Thursday, October 08, 2015

PeopleSoft Streams from Oracle University

In February of this year, Oracle University launched the PeopleSoft Learning Stream. Oracle's Learning Streams are short, educational vignettes. I was given the privilege of recording 6 streams:

  • Using JavaScript with Pagelet Wizard is a 21 minute video showing you how to use Pagelet Wizard to convert a PeopleSoft query into an interactive D3 chart, a navigation collection into a carousel, a navigation collection into an accordion, and RequireJS for JavaScript dependency management.
  • REST Query Access Service is a 15 minute session showing you how to craft a Query Access Service REST URL.
  • Working with JSON in PeopleSoft Document Technology is a 23 minute video demonstrating how to use the PeopleCode Document, Compound, and Collection objects to read and write JSON.
  • Basic Java API with PeopleCode is a 26 minute session showing you how to use the delivered Java API with PeopleCode. This session covers constructors, instance methods, properties, and static method invocation. Java objects demonstrated include String, Hashtable, Regular Expression Pattern and Matcher, arrays, and String.format.
  • Intermediate Java API with PeopleCode is a 38 minute video that shows you how to configure JDeveloper to write Java for the PeopleSoft Application and Process Scheduler servers and provides some examples of writing and deploying Java to a PeopleSoft application server. Note: in this session you get to watch me attempt to troubleshoot an App Engine ABEND.
  • Advanced Java API with PeopleCode is a 26 minute recording showing you how to use Java Reflection to remove PeopleCode ambiguity as well as how to use JavaScript to avoid reflection.

You can access all of my streams here. From this page you can preview the first 2 minutes of each video or subscribe for unlimited access to all of the videos in the Oracle PeopleSoft Learning Stream.

Wednesday, November 14, 2012

AWE Mass Approval

The sample chapter for my PeopleTools Tips and Techniques book (Chapter 3) contains all of the steps required to add AWE to a PeopleSoft transaction. A colleague who recently used this chapter to AWE enable a transaction asked me how to mass approve transactions. Most of the code required to mass approve transactions is actually on the last page of Chapter 3. Here is an expanded template with placeholders. Just wrap this in a loop and wire it up to a button, App Engine, or some other execution environment.

Local Record &headerRec = CreateRecord(Record.NAME_OF_AWE_HEADER_RECORD);  
Local EOAW_CORE:ApprovalManager &apprManager;  
Local string &processId = /* hard coded value goes here */;  

REM ** Populate approval header record keys here;
&headerRec.GetField(Field.KEY1).Value = /* Key 1 from scroll */
&headerRec.GetField(Field.KEY2).Value = /* Key 2 from scroll */
...

&apprManager = create EOAW_CORE:ApprovalManager(&processId, &headerRec, %OperatorId);  

If (&apprManager.hasAppInst) Then  
  &apprManager.DoApprove(&headerRec);  
Else  
  REM ** throw error;  
End-If;

Each time through the loop, update the header record values, acquire a new instance of the ApprovalManager, and execute DoApprove.

Friday, May 23, 2008

AppEngine Output Tricks, Reporting, Logging, Etc

Reporting

When I took the AppEngine course several years ago, my instructor made sure his students knew that AppEngine was a batch processing tool, not a reporting tool, unlike SQR, which could do both. At that time, PeopleSoft offered Crystal Reports, PS/nVision, PS Query, and SQR as reporting options, and he encouraged us to use those tools for reporting. AppEngine was strictly labeled a batch processing tool. While debugging some jobs containing AppEngines (dunning letters, training letters, etc), I noticed that these AppEngines created and/or read files from the process output directory. Looking at the process monitor, I knew that those same files were available from the View Log/Trace link on the process details page. This got me thinking... if I could create a Microsoft Word file from AppEngine, I could place it in that process output directory and not have to run the WINWORD process on a headless server. Now, the trick, creating a Microsoft Word file from an AppEngine... Here are a couple of options

  • Word HTML format
  • Word XML format
  • RTF

By using various methods, I can convert my mail merge source data into XML format and transform it into either of these three Microsoft Word recognized formats using XSL. Of course, as of PeopleTools 8.48, we can use XMLPublisher to generate the same result. Nevertheless, if you need to process your data prior to generating a report, then a multi-step AppEngine reporting solution might be easier for you to manage than a multi-step job.

The same options are available for creating Microsoft Excel and OpenOffice documents. If you want to create Microsoft Excel binary files from AppEngine, then you can use Apache's POI Java libraries from PeopleCode. If you are interested in creating OpenOffice documents, you can generate the appropriate XML files, and then use Java to zip them into a single file. In fact, you could use this same approach to generate OpenOffice Impress presentations or Microsoft PowerPoint 2007 presentations.

For reporting, why choose AppEngine over SQR? AppEngine components (PeopleCode, SQL, etc) are managed objects. PeopleTools managed objects participate in the change management features available in PeopleTools. SQR text files do not.

Can I create a PDF from an AppEngine? Yes. Using an XSL-FO processor, you can trasform XML into PDF using a user defined XSL template. Likewise, you can use one of the PDF Java libraries to print text to a PDF file using PeopleCode similar to the way you would print output to a PDF in SQR, but with rich text features. Other reporting options: Any reporting/output tool that has a Java API can be called from AppEngine PeopleCode. For example, JasperReports, BIRT, JFreeReport, FOP, etc.

File Output Location

Suppose I want to create a file (printable report, log file, etc), where should I create the file? If you want the file available from the View Log/Trace link, then use the following SQL to determine the process's output directory:

SELECT PRCSOUTPUTDIR FROM PSPRCSPARMS WHERE PRCSINSTANCE = %ProcessInstance

Logging

I've already mentioned using log4j from PeopleCode. You can read about that in my posts: Logging PeopleCode Using log4j to debug applications and log4j and PeopleCode Part II. Other options include the Peoplecode MessageBox function, the PeopleCode File object, and the Java System.out/System.err methods. I prefer the Java System output methods over the PeopleCode MessageBox function because Java gives me complete control over the output. Unfortunately, you can't call the Java System output methods directly because the PrintStream output methods are overloaded. Instead, we need to use reflection to call the print methods. Here are some functions you can place in a FUNCLIB that allow you to print to stdout and stderr from PeopleCode:

/*
* Print a line of text to stdout
*/
Function println_to_stdout(&message As string)
Local JavaObject &jSystem = GetJavaClass("java.lang.System");
Local JavaObject &jOutStream = &jSystem.out;
Local JavaObject &jCls = GetJavaClass("java.lang.Class");
Local JavaObject &jStringClass = &jCls.forName("java.lang.String");
Local JavaObject &jPrintStreamCls = &jOutStream.getClass();
Local JavaObject &jPrintlnArgTypes = CreateJavaObject("java.lang.Class[]", &jStringClass);

Local JavaObject &jPrintlnMethod = &jPrintStreamCls.getDeclaredMethod("println", &jPrintlnArgTypes);

&jPrintlnMethod.invoke(&jOutStream, CreateJavaObject("java.lang.Object[]", &message));
rem ** I didn't find flushing necessary, but here is where you would flush the buffer if desired;
rem &jOutStream.flush();
End-Function;

/*
* Print a line of text to stderr
*/
Function println_to_stderr(&message As string)
Local JavaObject &jSystem = GetJavaClass("java.lang.System");
Local JavaObject &jOutStream = &jSystem.err;
Local JavaObject &jCls = GetJavaClass("java.lang.Class");
Local JavaObject &jStringClass = &jCls.forName("java.lang.String");
Local JavaObject &jPrintStreamCls = &jOutStream.getClass();
Local JavaObject &jPrintlnArgTypes = CreateJavaObject("java.lang.Class[]", &jStringClass);

Local JavaObject &jPrintlnMethod = &jPrintStreamCls.getDeclaredMethod("println", &jPrintlnArgTypes);

&jPrintlnMethod.invoke(&jOutStream, CreateJavaObject("java.lang.Object[]", &message));
rem ** I didn't find flushing necessary, but here is where you would flush the buffer if desired;
rem &jOutStream.flush();
End-Function;

If you want to use the PrintStream.print method instead of the println method, copy the code above, rename the function, and change the &jPrintlnMethod assignment from "println" to "print".

If you've worked with Java, then you know that you can redirect stdout and stderr to another PrintStream. For example, you can redirect stdout to a file or a network socket connection. Here is some code demonstrating how to redirect stdout and stderr to a different file:

/*
* Redirect stdout to file
*/
Function redirect_stdout(&fileName as string)
Local JavaObject &jSystem = GetJavaClass("java.lang.System");
Local JavaObject &jfos_out = CreateJavaObject("java.io.FileOutputStream", &fileName, True);
Local JavaObject &jps_out = CreateJavaObject("java.io.PrintStream", &jfos_out, True);
&jSystem.setOut(&jps_out);
End-Function;

/*
* Redirect stderr to file
*/
Function redirect_stderr(&fileName as string)
Local JavaObject &jSystem = GetJavaClass("java.lang.System");
Local JavaObject &jfos_out = CreateJavaObject("java.io.FileOutputStream", &fileName, True);
Local JavaObject &jps_out = CreateJavaObject("java.io.PrintStream", &jfos_out, True);
&jSystem.setErr(&jps_out);
End-Function;

By redirecting stdout and stderr, you could actually create 3 separate output files without using the File object. The benefit of using a redirected stdout over a File object is that you can setup your stdout location in one step of your program and write to that same file from anywhere else in the program without having to open/close a File object on every step.

The App Server

Just a side note: Many of the techniques demonstrated in this post can be used online. Using System.out.println, you could print to the app server's stdout file. Likewise, the reporting solutions above could be used from an online PeopleCode event to generate reports online.