Can you post a minimal workspace here that shows the HTTPCaller and all the associated settings?
Hi @david_r , I have just uploaded a HTTPCallerDummy workspace so you can see the settings I currently have.
Out of curiosity why are you using the HTTPCaller instead of the FMEServerJobSubmitter?
Out of curiosity why are you using the HTTPCaller instead of the FMEServerJobSubmitter?
Hi @jdh, I am doing so because I need to retrieve the output of the workspace I run which I believe is not possible by using FMEServerJobSubmitter. Correct me if I am wrong. Using the HTTPCaller I am able to retrieve the output as JSON and continue the data transformation process.
Hi @jdh, I am doing so because I need to retrieve the output of the workspace I run which I believe is not possible by using FMEServerJobSubmitter. Correct me if I am wrong. Using the HTTPCaller I am able to retrieve the output as JSON and continue the data transformation process.
You may want to either use, or let yourself be inspired by, the FMEServerDataDownload custom transformer available on the FME Hub:
https://hub.safe.com/publishers/giosp/transformers/fmeserverdatadownload
Hi @jdh, I am doing so because I need to retrieve the output of the workspace I run which I believe is not possible by using FMEServerJobSubmitter. Correct me if I am wrong. Using the HTTPCaller I am able to retrieve the output as JSON and continue the data transformation process.
thanks @david_r, I am calling this as Data Streaming Service though. So not sure if this would still help me but I will take a look definitely.
Is there any way though I could use HTTPCaller but passing the attributes as part of the body to make the POST request?
As mentioned before, I have uploaded a workspace dummy with the parameters I am using to make the call. So maybe this gives you the hint as to what I am doing wrongly.
Hi @jdh, I am doing so because I need to retrieve the output of the workspace I run which I believe is not possible by using FMEServerJobSubmitter. Correct me if I am wrong. Using the HTTPCaller I am able to retrieve the output as JSON and continue the data transformation process.
Not directly no, the FMEServerJobSubmitter only returns the job ID. The FMEServerLogFileRetriever will return the log file which can be parsed to get the output location, but at that point you probably are better using the rest api directly instead.
I believe that when you invoke the fmedatastreaming service, the message body of your POST request is sent as input to the workspace reader, and not automatically parsed as published parameters (as is the case for the "regular" REST API).
See: https://docs.safe.com/fme/html/FME_Server_Documentation/ReferenceManual/service_datastreaming.htm
This means that you'll have to use a Textfile reader in your workspace to pick up the message body contents, then something like the JsonFlattener to get the srid and wkt attributes in the JSON you're posting. Alternatively you could also use a JSON reader (but personally I find it easier to debug with a Textfile reader).
I believe that when you invoke the fmedatastreaming service, the message body of your POST request is sent as input to the workspace reader, and not automatically parsed as published parameters (as is the case for the "regular" REST API).
See: https://docs.safe.com/fme/html/FME_Server_Documentation/ReferenceManual/service_datastreaming.htm
This means that you'll have to use a Textfile reader in your workspace to pick up the message body contents, then something like the JsonFlattener to get the srid and wkt attributes in the JSON you're posting. Alternatively you could also use a JSON reader (but personally I find it easier to debug with a Textfile reader).
Thanks @david_r this might be the reason. However, how is it that Postman is able to replicate the same call I make with the HTTPCaller and retrieve the workspace output whereas FME cannot? Postman is calling Data Streaming Service as well so I believe there might be a way to replicate that on FME without the need of modifying the workspace that get called something which I would like to avoid as these data streaming workspaces are being called from various side, not only FME.
Thanks @david_r this might be the reason. However, how is it that Postman is able to replicate the same call I make with the HTTPCaller and retrieve the workspace output whereas FME cannot? Postman is calling Data Streaming Service as well so I believe there might be a way to replicate that on FME without the need of modifying the workspace that get called something which I would like to avoid as these data streaming workspaces are being called from various side, not only FME.
How did you configure the request in Postman compared to the HTTPCaller? Could you post a screenshot of the Postman setup?
Thanks @david_r this might be the reason. However, how is it that Postman is able to replicate the same call I make with the HTTPCaller and retrieve the workspace output whereas FME cannot? Postman is calling Data Streaming Service as well so I believe there might be a way to replicate that on FME without the need of modifying the workspace that get called something which I would like to avoid as these data streaming workspaces are being called from various side, not only FME.
Same as this one @david_r
Thanks @david_r this might be the reason. However, how is it that Postman is able to replicate the same call I make with the HTTPCaller and retrieve the workspace output whereas FME cannot? Postman is calling Data Streaming Service as well so I believe there might be a way to replicate that on FME without the need of modifying the workspace that get called something which I would like to avoid as these data streaming workspaces are being called from various side, not only FME.
Thanks, I see the difference. For the same setup as in Postman, you'll have to configure the HTTPCaller with upload data set to "Multipart / Form Data", then specify the key/value pairs in the "Multipart Upload" pane below:
Thanks @david_r this might be the reason. However, how is it that Postman is able to replicate the same call I make with the HTTPCaller and retrieve the workspace output whereas FME cannot? Postman is calling Data Streaming Service as well so I believe there might be a way to replicate that on FME without the need of modifying the workspace that get called something which I would like to avoid as these data streaming workspaces are being called from various side, not only FME.
Thanks @david_r but I tried that before making this enquiery and it didn't work. So that is way I approached the idea of sending it as Json within the Upload Body.
Thanks @david_r this might be the reason. However, how is it that Postman is able to replicate the same call I make with the HTTPCaller and retrieve the workspace output whereas FME cannot? Postman is calling Data Streaming Service as well so I believe there might be a way to replicate that on FME without the need of modifying the workspace that get called something which I would like to avoid as these data streaming workspaces are being called from various side, not only FME.
What error message do you get?
Thanks @david_r this might be the reason. However, how is it that Postman is able to replicate the same call I make with the HTTPCaller and retrieve the workspace output whereas FME cannot? Postman is calling Data Streaming Service as well so I believe there might be a way to replicate that on FME without the need of modifying the workspace that get called something which I would like to avoid as these data streaming workspaces are being called from various side, not only FME.
@david_r, not an error at all. I am able to run the workspace but still having the issue where no attributes is being passed through the call. So I have my custom JSON response telling me it has not valid WKT or Order Id and therefore the workspace is terminated.
The below is how I have tried. Also using using application/x-www-form-urlencoded which is the content type my Postman call is using.
First: Upload Data in the HTTPCaller should be set to "Specify upload body", not "Multipart / form data", sorry for the confusion.
Secondly: The most important thing to know is that there is no automatic mapping between the upload body and any published parameters in the workspace when using the fmedatastreaming service. This is really important, because the only way to retrieve the contents from the upload body is through a text (or similar) reader in the workspace that is being called. This is also mentioned in the documentation that I linked somewhere else in this thread.
First: Upload Data in the HTTPCaller should be set to "Specify upload body", not "Multipart / form data", sorry for the confusion.
Secondly: The most important thing to know is that there is no automatic mapping between the upload body and any published parameters in the workspace when using the fmedatastreaming service. This is really important, because the only way to retrieve the contents from the upload body is through a text (or similar) reader in the workspace that is being called. This is also mentioned in the documentation that I linked somewhere else in this thread.
Hi @david_r, you are absolutely right. Somehow I thought I could replicate what Postman was doing but I was wrong.
Yesterday, I started investigating further the information you sent and I was able to solve the issue by just adding a Text File Reader within the FME Workspace I was planning to make the POST Call.
So many thanks for your help and time :)
Hi @david_r, you are absolutely right. Somehow I thought I could replicate what Postman was doing but I was wrong.
Yesterday, I started investigating further the information you sent and I was able to solve the issue by just adding a Text File Reader within the FME Workspace I was planning to make the POST Call.
So many thanks for your help and time :)
Am glad to hear! Regarding Postman, one possibility could be that there were default values defined in the published parameters.
