My blog contains a handful of posts showing various methods for base64 encoding data (both plain text and binary). While these procedures apply universally, my primary motivation was (and is) integration: sending base64 encoded binary data to other systems. Today I was looking through the 8.52 Integration Broker PeopleBooks and noticed a handful of new features to support binary file operations:
- MTOM Support for sending and receiving binary data through Integration Broker.
- File.GetBase64StringFromBinary() for reading a binary file into a base64 text string. Please note: the documentation says it reads the "complete contents of the binary file" into a string. You will want to make sure you don't execute this method against files that are too large for your app server's memory constraints.
- File.WriteBase64StringToBinary().
What about files uploaded by users? How do you work with uploaded files? Uploaded files are stored as file attachments in either a database record or FTP destination. The File Attachment API provides the GetAttachment and PutAttachment PeopleCode functions to help you move files in and out of file attachment repositories.
As of 8.52, it seems we no longer need to use the Apache Commons Codec Java library to convert binary files into base64 or base64 into binary files. Likewise, prior to 8.52, if someone asked me if PeopleCode could create a binary file, I would say, "No, but you can use the Java API classes in PeopleCode to write binary files." Now I can say, "Yes, but you will first have to convert your binary data to base64." Hmmm... still sounds a bit funny. Is it better? I think so. Reading and writing binary was the hard part. With 8.52, I have that functionality without having to maintain an extra Java library in my class path. Besides pluggable encryption, as I and my readers demonstrated in my post Base64 Encoding for PeopleSoft, there are a number of ways to encode data including using database features and delivered Java methods (for SQL Server, see sachin's comments here and here).
28 comments:
Jim - I have an unrelated question you may be able to answer. What's the best way to manage memory usage when using, app engines which calls an app package, in turn which uses other app packages and CI's? Cancel the CI after each save and then reinstantiate it? Keep the original instance of the CI and repeatedly do new gets/creates (I don't even know if you can do this?). I guess my bigger question is how does memory usage work with these objects and how is it cleaned up? I'm basically running a custom process and it's running out of memory after 10,000 rows. But, I'm cancelling the CI after each row, the original data is in a 20,000 row rowset. Thanks
@Rbop51, that is a good question. You may want to post it to the OTN PeopleSoft forum. From what you are seeing, it does appear that the component buffers are staying in memory and not getting cleaned up. I thought you had to create a new CI instance for each instance. Once you call get or create, I think you are done with that one and have to start over. I could be wrong though.
Perhaps another approach would be to create a Job that has a couple of app engines, and each one uses a different segment of the data? This assumes that you can partition the data and treat it as subsets of data.
Thanks Jim, will do. I haven't fully analysed it yet, but much to my surprise it's not blowing up when I use the same CI instance over and over and do a new get. Doesn't mean that it's working correctly though. I'll keep you posted.
Jim - You are correct it doesn't work if you leave the CI instantiated. I solved my problem by throwing the kitchen sink at it. I added commitwork(), collectgarbage() and cancel after each row and it ran through them all no problem. Also it doubled the speed.
Thanks,
Russ
@Russ, that is very good to know. I appreciate the response. Cutting the run time in 1/2 is very impressive.
How are you handling restart if it fails? Do you mark rows as processed so they won't be queried and reprocessed?
I wonder if you could fine tune it even more by only calling CommitWork and CollectGarbage every 500 or 1,000 rows or so? This would cut down on disk writes and may reduce memory operations, but not sure about that one. Not sure if it is worth the effort to investigate.
Just FYI, I've seen similar issues with Integration Broker and CI's. I suspect the same solution would help with IB memory issues as well.
For other readers of these comments, you can view additional comments on Russ's OTN thread.
What if you are needing to use Base64 dynamically, like as part of a web service? Leveraging some comments from your blog we came up with this, which is testable with the App Designer debugger:
Local string &myEncodedString, &plainTextXMLData, &myDecodedString;
Local JavaObject &EncodeDecode, &StringObj, &ByteObj;
Local boolean &bResult;
&plainTextXMLData = "Input sample Text.";
&EncodeDecode = CreateJavaObject("com.peoplesoft.tools.util.Base64");
/*Encode*/
&ByteObj = CreateJavaObject("java.lang.String", &plainTextXMLData).getBytes();
&myEncodedString = &EncodeDecode.encode(&ByteObj);
/* Decode*/
&StringObj = CreateJavaObject("java.lang.String", &EncodeDecode.decode(&myEncodedString));
&myDecodedString = &StringObj.toString();
@doubleDog, right, if you are working with non-file data, for example, strings, then, yes, you will still need to use one of the approaches described in my other blog posts. Your code is a good example. Besides the undocumented com.peoplesoft.tools.util.Base64 method, you may also want to look into the documented Pluggable Encryption method demonstrated in this blog post. The pluggable encryption route requires a bit more configuration, though.
HI, Can any one provide me inputs on the base64 encoding in peoplesoft. Our users upload the files online to FTP , we are encoding these files and sending to Filenet system.
Currently we are at 8.49 tools, we are going to move to 8.52 tools soon.
Will there be any difference for encoding for these versions.
Thanks.
@Sri, PT 8.52 has some new methods that make Base64 encoding easier, but the base64 encoding mechanisms used for 8.49 will continue to work in 8.52. For releases prior to 8.52, I recommend this approach which uses Java Objects to read binary files and convert them to base64 text. Once converted, you can insert them into XML messages. My Base64 Encoding Binary Files post uses the apache commons Base64InputStream because it is easier, but if you can't use 3rd party libraries, you can still accomplish this using delivered base64 encoding libraries. See the comments in this post for details.
Jim,
We got one scenario. In one of the integration with third party using WSDL we are getting response with attachment. Below is the raw XML response (Using SOAPUI). We are not sure how to handle this RESPONSE in PS. Please share ur thoughts.
HTTP/1.1 200 OK
Date: Mon, 08 Oct 2012 06:46:54 GMT
Server: Apache
Cache-Control: no-store
Transfer-Encoding: chunked
Content-Type: multipart/related; type="text/xml"; boundary="uuid:b4ee8c4f-7749-4146-a0c8-42ac06e19046"
--uuid:b4ee8c4f-7749-4146-a0c8-42ac06e19046
Content-Type: text/xml
--uuid:b4ee8c4f-7749-4146-a0c8-42ac06e19046
Content-Id:
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
"Application Id","Template Id","Country","Job Code"
"241","262","","123"
"241","301","","123"
@mymithraa, the blog comments editor does not allow you to paste in XML directly, so it is difficult to see the issue here. I suggest that you post your question on the OTN forum or in ITToolbox.
Hey Jim: Here is a cool way to use the 8.52 function GetBase64StringFromBinary(). We create a graphical calendar for students via a java program and the end result is always a jpeg that we display directly on a page. We have used several methods over the past few years, but I found a really cool way to display this without needing an iscript. Here is the gist:
Assuming you have your jpg in a file on the app server:
Local File &imgFile = GetFile(&str_FileName, "R", "", %FilePath_Relative);
DERIVED.HTMLAREA1 = "<... src=""data:image/jpeg;base64," | &imgFile.GetBase64StringFromBinary() | """/>";
That's it! The image is base64 encoded and then the contents pushed directly to the browser.
The issue I had with using an iscript was the image was dynamic and one-time, so if I deleted it from the database before the page loaded, the image was broken.
Regards,
John
@John, thanks! Nicely done! Great use of the File.GetBase64FromBinary method.
Jim - I am working on an 8.52 peopletools system and I am using integration broker to communicate with a mobile device. Up until now, all of the development has been with a REST Service. I now need to be able to recieve attachments from the mobile device that will primarily be images.
My question is if it is possible to do this with a REST service using message segments/MTOM-encoded? I have been looking through peoplebooks and there seems to be an indication that it's possible, but REST services require the REST connectors and MTOM-encoding/message segments require either HTTP Connectors or PeopleSoftServices Connectors.
Thanks in advance for any insight you can shed on this subject.
Kyle
@Kyle, is this a web based app or a native app? Are you using Canvas or file uploads or a native app with a REST service? I assume you are using PUT or POST? With Canvas or a native app, I recommend base64 encoding the file to upload (canvas supports this). Then you can place the base64 directly in your text-based payload. I am not sure how to handle this with a file upload, that is why I make the distinction.
MTOM... I actually have no experience with MTOM yet. You might post the same question on the OTN forum
Jim - This is a native app and I am using GET/POST to send/receive data from it. Thanks for the suggestion, this is currently what I have in place.
I've been looking into message segments because it appeared to possibly be more efficient but so far it has just appeared to be using everything we aren't.
I will follow up on OTN and see if there are any suggestions there.
Thanks,
Kyle
Hi Jim,
I'm using WriteBase64StringToBinary to write a file on the App server, then I need to display it to the user with ViewAttachment.
is it possible to use "file://" instead of "ftp://" on Linux/PeopleTools 8.52 to get the file ?
Thanks.
@Stéphane, file:// is for reading local files and is from the perspective of the browser. It won't transfer files between the server and the client. If you write the file into the HTTP path, then you can use http(s)://, but that may not be safe depending on data security. The safest way I know to transfer binary data from the server to the client is through an iScript with %Response.WriteBinary. The problem with this approach is that selecting binary from the database into a variable of type "Any" is the only way I know to get binary data into a variable.
Jim, In fact I tryed an IScript with this:
Local File &F1;
Local string &base64string;
&F1 = GetFile(&filename, "R", %FilePath_Absolute);
If &F1.IsOpen Then
&base64string = &F1.GetBase64StringFromBinary();
&F1.Close();
End-If;
%Response.Write(&base64string);
but it does not work as the downloaded filesize is 80k instead of 100k ...
Any idea ? I can't file solution in your books :)
Jim,
regarding the file:// topic, the idea was to use it with the ViewAttachement function, but I can't make it work on UNIX.
/Stéphane.
@Stéphane, your %Response.Write would write the base64 content. You want the binary content. I was referring to this blog post. Notice that I select from a database table, not read from a file. That is because there is no way to get binary content from a file into a variable (there might be, I just haven't found it).
Hi Jim,
Thanks for your very valuable help.
I fact I was using a record to store a temp file doing:
PutAttachement
ViewAttachement
DeleteAttachement
But it does not work may be due to the lantency with the browser as it gets deleted before it's opened in the browser ... any clue ?
So finaly I tryed your proposal
and I'm doing:
PutAttachement
Then Call an Iscript where I use your method:
%Response.WriteBinary(&data);
and finally DeleteAttachement in the IScript
So for now I'm happy that it works, but I wonder why it does not work with the full Attachement API.
best regards,
/Stéphane.
Hi Jim,
We wanted to access the SharePoint Portal from PeopleSoft
@Mohd, there are a good 1/2 dozen ways to base64 encode data using PeopleTools. This blog post contains links to those other methods. For zip files, a couple of years ago I wrote the post Manipulating Zip Files with PeopleCode. If you are using PeopleTools 8.53, zip files are even easier with the Java ZipFileSystem, but the older method still works.
Hi Jim,
Hope you're doing great.
I am using ftp to store file attachments on server, however now I need to store Image to Ftp server. I receive this image in binary format from 3rd party java application. Is there a way through which I can convert this binary image data stored in database to a file on some temporary location on the server. This file in temporary location then can be used to attachment functionality which is already existing.
Thanks in advance.
@prasad.poojari, there are a couple of ways. You can use tables as targets for the File Attachment API. If your images are in a record that fits the format, you can use GetAttachment. If not, then you may need to use database support for exporting files (PL/SQL, etc).
Hi Jim,
I'm working on consuming a webserive where the attachments in 3rd party system are sent in base 64 format in the response message. I receive a successful base 64 response when I make a call to the API from tools like Postman or RESTClient but I'm receiving an empty response when tried from Peoplesoft. Can you please help?
Thanks
Post a Comment