Wednesday, February 24, 2021

Setting IB Request Content Type

In our Integration Day 2021 One-day Webinar, we showed how to send SMS from PeopleSoft through Twilio. What makes Twilio interesting is that Twilio expects URL encoded form data. As you may know from using Integration Broker, PeopleSoft is quite happy sending and receiving JSON or XML. But what about other content types, such as application/x-www-form-urlencoded? You may know that you can set most HTTP headers using something like:

&request.IBInfo.IBConnectorInfo.AddConnectorProperties("Content-Type", "application/x-www-form-urlencoded", %Header);

That's a mouth full! But the weirdest thing happens when trying that trick with URL encoded form data. PeopleSoft URL encodes the entire body! Wait, isn't that correct? No, actually. When you and I create the body, we create the proper key=value&key2=value2 pairs. Proper URL encoded form data would look something like this:

key=This+is+some+encoded+data&key2=value2

Keys, ampersands, and equals signs should not be encoded (unless part of the value). But PeopleSoft URL encodes everything as if there were no keys. Here is what that same string above looks like when PeopleSoft finishes with it:

key%3DThis%2Bis%2Bsome%2Bdata%2Bto%2Bencode%26key2%3Dvalue

It is just supposed to URL encode the values. Or better yet, just let us URL encode the values. What's the workaround?

&request.SegmentContentType = "application/x-www-form-urlencoded";

This solution is a little strange because we aren't using a multi-part, segmented message, but it works!

While performing research for this post, I came across MOS Doc ID 2187249.1, which asks for an enhancement to allow additional, hopefully even custom, Content Types, so it appears URL encoded form data isn't the only issue. But lucky for us, we have SegmentContentType as a workaround!

Are you interested in learning more about PeopleSoft Integration Broker and integration strategies? Join one of our live virtual top-rated Integration Tools Update courses!

13 comments:

Anirudh said...

I believe we can also use &MSG.IBInfo.IBConnectorInfo.AddQueryStringArg as a work around for the encoding issue. this way fields remain as separate values and not as a single string.

Sabarna said...

Using AddQueryStringArg, will work only with certain applications. not all apps accept content in query string. for e.g., Azure expects the content to be in the body.

I tried setting request content type for azure AD Oauth(client credentials grant type), it works properly until the message leaves application server, but in target connector log, the body content is fully encoded. I found this from IB msglog.

Andy said...

Hi Jim,
If I'm not mistaken, Twilio appears to use something called an Account SID as the username for basic authentication to their API. It's 34 characters.

When you put in the request headers, the Basic Authentication username field (IBEXTERNALUSERID) only allows 30 characters.
Did you happen to come across this? And wondering if you have any ideas?

Jim Marion said...

@Andy, correct. Yes, we ran into this as well. Here is a PeopleCode fragment for setting the Authorization header:

&request.IBInfo.IBConnectorInfo.AddConnectorProperties("Authorization", "Basic " | &token, %Header);

You are responsible for base64 encoding the username:password, which is &token above. We have several examples of base64 on our blog. I recommend the PET example.

Andy said...

Thanks, Jim. That helped us get past the authorization header issue. But getting stumped by something else now where the gateway logs are showing "A 'To' phone number is required..."

My REST based URL looks like this:
https://api.twilio.com/

And my URI template with Index 1 looks like this:
2010-04-01/Accounts/{acc_SID}/Messages/
(filling the acc_SID with my document template)

To pass the To phone number and other parameters in the body as x-www-form-urlencoded, I'm populating the request message:

&MSG = CreateMessage(Operation.SENDSMS_GET);
//code that populates URI document template

//populate request message
&DOC = &MSG.GetDocument();
&COM = &DOC.DocumentElement;
&COM.GetPropertyByName("To").Value = "+12222222222";
&MSG.SegmentContentType = "application/x-www-form-urlencoded";

&RESP = %IntBroker.SyncRequest(&MSG);
MessageBox(0, "", 0, 0, "%1", &RESP.genxmlstring());

But seeing this in the gateway logs "A 'To' phone number is required..."
No issues when sending it the same way in PostMan. Wondering if you have any ideas?

Jim Marion said...

Hi Andy,

The problem is that you are using a document and creating XML. The message body is supposed to be URL Encoded Form data. Documents don't support URL encoded form data. Your request message for the post should be unstructured. I like using IB_GENERIC_REST.

I recommend checking out our Integration Day 2021 replay video as it walks you step-by-step through Twilio integration and includes sample source code you can alter to satisfy your requirements. You can find the replay online at https://learn.jsmpros.com/courses/id21.

Rafa Barcelo said...

Hi Jim.

Do you know how to make this with form-data?

Thank you

Jim Marion said...

@Rafa, that is what this post describes. Form data is URL encoded.

Kesav said...

Thanks, Jim. Somehow this code is not working for me. I'm using the delivered Generic Service Operation to send parameters as string. Receiving system is throwing grant_type missing error. Not sure something I'm missing

Request Content-Type = application/x-www-form-urlencoded
Response Content-Type = application/json


&oGenRqstRESTMsg_ = CreateMessage(Operation.IB_GENERIC_REST_POST);
Local IBInfo &IBInfo_ = &oGenRqstRESTMsg_.IBInfo;
Local boolean &bRet = &IBInfo_.LoadConnectorProp("HTTPTARGET");
&IBInfo_.ConnectorOverride = True;
&IBInfo_.IBConnectorInfo.ConnectorClassName = "HttpTargetConnector";
&IBInfo_.IBConnectorInfo.ConnectorName = "HTTPTARGET";
&sUrl_ = "https://test.org/abc/connect/token";
&sAction_ = "POST";
&bRet = &IBInfo_.IBConnectorInfo.DeleteConnectorProperties("Content-Type");
&bRet = &IBInfo_.IBConnectorInfo.AddConnectorProperties("Content-Type", "application/x-www-form-urlencoded", %HttpHeader);


&bRet = &IBInfo_.IBConnectorInfo.DeleteConnectorProperties("URL");
&bRet = &IBInfo_.IBConnectorInfo.AddConnectorProperties("URL", &sUrl_, %HttpProperty);
&bRet = &IBInfo_.IBConnectorInfo.DeleteConnectorProperties("Method");
&bRet = &IBInfo_.IBConnectorInfo.AddConnectorProperties("Method", &sAction_,
%HttpProperty);

&sRequest_ = "grant_type=client_credentials&client_id=XX&client_secret=XXXXX-DEV";

&bRet = &oGenRqstRESTMsg_.SetContentString(&sRequest_);
&oGenRqstRESTMsg_.SegmentContentType = "application/x-www-form-urlencoded";

&oGenRspnsRESTMsg_ = %IntBroker.ConnectorRequest(&oGenRqstRESTMsg_, True);

Kesav said...

Thanks, Jim. Somehow this code is not working for me. I'm using the delivered Generic Service Operation to send parameters as string. Receiving system is throwing grant_type missing error. Not sure something I'm missing

Request Content-Type = application/x-www-form-urlencoded
Response Content-Type = application/json


&oGenRqstRESTMsg_ = CreateMessage(Operation.IB_GENERIC_REST_POST);
Local IBInfo &IBInfo_ = &oGenRqstRESTMsg_.IBInfo;
Local boolean &bRet = &IBInfo_.LoadConnectorProp("HTTPTARGET");
&IBInfo_.ConnectorOverride = True;
&IBInfo_.IBConnectorInfo.ConnectorClassName = "HttpTargetConnector";
&IBInfo_.IBConnectorInfo.ConnectorName = "HTTPTARGET";
&sUrl_ = "https://test.org/abc/connect/token";
&sAction_ = "POST";
&bRet = &IBInfo_.IBConnectorInfo.DeleteConnectorProperties("Content-Type");
&bRet = &IBInfo_.IBConnectorInfo.AddConnectorProperties("Content-Type", "application/x-www-form-urlencoded", %HttpHeader);


&bRet = &IBInfo_.IBConnectorInfo.DeleteConnectorProperties("URL");
&bRet = &IBInfo_.IBConnectorInfo.AddConnectorProperties("URL", &sUrl_, %HttpProperty);
&bRet = &IBInfo_.IBConnectorInfo.DeleteConnectorProperties("Method");
&bRet = &IBInfo_.IBConnectorInfo.AddConnectorProperties("Method", &sAction_,
%HttpProperty);

&sRequest_ = "grant_type=client_credentials&client_id=XX&client_secret=XXXXX-DEV";

&bRet = &oGenRqstRESTMsg_.SetContentString(&sRequest_);
&oGenRqstRESTMsg_.SegmentContentType = "application/x-www-form-urlencoded";

&oGenRspnsRESTMsg_ = %IntBroker.ConnectorRequest(&oGenRqstRESTMsg_, True);

Jim Marion said...

@Kesav, there may be other issues, but unfortunately, you can't use AddConnectorProperties for URL encoded form data. You have to use SegmentContentType like this: &oGenRqstRESTMsg_.SegmentContentType = "application/x-www-form-urlencoded";

Kesav said...

Thank you! I've opened SR with Oracle based on the input from support engineer, PS urlencoded string different than other languages. Not sure who is following urlencoded standards correctly . Example is below

&sRequest_ = "grant_type=client_credentials&client_id=XX&client_secret=XXXXX-DEV";

PS urlencoded (using the https://www.urlencoder.org/)
grant_type%3Dclient_credentials%26client_id%3DXX%26client_secret%3DXXXXX-DEV

Postman/3rd Party: not encoding &, = s. Only special char like ! encoded
grant_type=client_credentials&client_id=XX&client_secret=XXXXX-DEV

TO avoid this we have moved these to headers which is working fine.

Jim Marion said...

@Kesav, another troubleshooting idea is to setup a Man in the Middle proxy to review what PeopleSoft is actually sending. I have a video that shows how to do this: https://youtu.be/stVGhU8AR5c