best method email processing fgdb

Hi @gisbradokla, 

Good question! 

To modify the workspace to accept the file as an attachment. To do so you’ll want to remove the path reader, sorter and sampler and replace it with a creator. Then, set the FeatureService reader to read in the email attachment as a user parameter. 

The pros of this are the attachment will be processed immediately. The cons would be the file would only be stored temporarily and removed. So, if something went wrong with the processing and you didn’t notice it for a while, it will be harder to retry the workspace. 

Either way works but I believe processing the attachment from the email trigger will be the quickest method. You can always download email attachment locally in that same workspace using the FMEFlowResourceConnector if you’d like a copy. 

thank you @siennaatsafe  . i am sorry i don’t quite understand.

what type of user parameter am i using to read the email attachment?

in the automation i have part of the email attachment filename as a filter to only allow this email to be processed. 

but what should i use in the featurereader for a filename? Also i am totally lost on how the flowresource connector would work to save a copy. previously i have used the filecopier.

The email is coming from an external customer to an imap email account.

Hi Brad, 

You can create the user parameter directly in the FeatureReader. 

From the email action you can connect the email action directly to the workspace, then use the Email Attachment. 

You can test this out in a new automation if you don’t want to alter your existing one. You shouldn’t need that filter if you are reading in the attachment from the email. 

As for the FMEFlowResourceConnector, I actually don’t think you need it. In the email trigger, you can set the Download Attachments To parameter to store the files, if you’d like to keep attachments around. Sorry, I forgot about this setting in my original post.

See https://docs.safe.com/fme/html/FME-Flow/WebUI/Automations-Triggers/Trigger-Email-IMAP.htm for details. 

@siennaatsafe I was finally able to get back to this and attempt to finish. 

I have the automation built and still having difficulty. 

It is quite hard to test as i am not aware of a way to pass an email into form.

that being said. 

I have the email attachment as the trigger (one other snag is) that our IT does not allow the use of the internal server email so we do have an alias set up (that does work with one other automation i have receiving email from that alias)

I send an email with attachment and nothing ever happens. i can see that the email is received but not sure how to troubleshoot after that. 

so if i have a filter on my other automation. do all emails go through any other atuomations that do not meet that filter? do i need to do something on that automation to get the email that doesn’t meet that filter to leave and get tested in the filter on this automation?

@gisbradokla, 

How is the other automation set up? Is the filter set in the email parameters or is it a Filter action in the automation?

@siennaatsafe 

on the other automation there is no filter (subject filter) on the email. it looks just like your attachment.

on the new automation I do have a subject filter of “dump” as the email looks like this. “FW: Weekly Data Dump - 10/7/2024”

now i am having difficulty with the workbench because i am trying to fan out the dwg file names by the feature data set.

So with the restrictions of the featureReader if i set it to my user parameter it says it cannot find the dataset to spread the featureclasses and featuredatasets out

If i allow only one port and try to set the writer to dynamic (in order to get all the attributes written to each feature “layer” in the OD autocad map dwg it does not work. (of course i am unable to test this in flow as per the other issuewe are discussing. and if i try to test in form i have to set the path in order to read in the gdb.zip file as it would be in the email.

Hi @gisbradokla,

I’d try setting a simple filter in the trigger of the other automation as well. I think what might be happening is the trigger without the filter, is reading all of the emails, therefore the other trigger is not receiving the emails.

As for your other issue. It’s hard to say what’s occurring without seeing the full workspace and logs. However, have you tried using a parameter fetcher to turn the parameter into an attribute before setting it as the dataset in the feature reader?

i keep hitting dead ends!

So the simple filter will only allow me to filter on the subject. but the subject in these email can change.

I have a filter next to the email trigger that filters on a specific term in the body. and if it is not there it goes to another filter for a second automation. I don’t really want to but the only thing i can think of. is to put a third workspace in the automation to allow the filter to fail twice and then trigger this one.

sets a precedence i don’t want to start.

That did not work anyway.

Hi Brad, 

I'm sorry to hear you're feeling stuck! There's almost always a solution to these challenges—it sometimes just takes a bit of creativity.

What about using the regex filter on the original automation? You could set it to read in all emails except the Weekly Data Dump. 

I asked ChatGPT about this and it gave me this answer:

^(?!.*\bWeekly Data Dump\b).+$

. You may have to refine this to find a statement that works for you but this is the explanation it gave me:
 

Thank you

i will give that a go. 
I have tried some AI to get regex with mixed results 

I hope it does work on this example. 
I did try stringing the filters together but that didn’t seem to work.