Hello All,
Is it possible to perform any sort of web crawling via FME? I have a button the website which downloads documents for me. I wish to automate it by some sort of crawling from FME which reaches the site and downloads the data.
Kindly help.
+49- Celebrity
You can use an HTTPCaller to download the web page and then parse through it. Depending on how it's built this may or may not give you access to the data. Plain HTML usually works ok but dynamic stuff can be a hassle.
+43- Safer
A link/URL is fairly simple, but to replicate the action of a button is trickier. Take a look at the source code for the page, then see if the HTTPCaller can be used to replicate it. Like @redgeographics says, you want to hope that they are using HTML and the button has a formaction and formmethod attribute. That you can easily mimic in a HTTPCaller. But if the site uses Javascript or similar then I don't know what you would do.
+17- Supporter
Just as @mark2atsafe says replicating the action of a button can be tricky. Using developers tools of your browser of choice might provide some insights on what is happening under the hood.
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.