Question

How to get information from website


Badge

Hi I want to send a number of Postcodes e.g. 1000-002 to this website:

http://www.ctt.pt/feapl_2/app/open/postalCodeSearch/postalCodeSearch.jspx#addressSearchPanel

 

and return the addresses in the e.g.

Rua dos Açores Ímpares de 23 a 47 Lisboa

Is this possible with FME? If so any suggestions how to set this up? Many Thanks.


4 replies

Userlevel 4
Badge +25

If they offer an API you would be able to use the HTTPCaller in FME to send requests. I can't seem to find one though (but I don't speak Portugese)

Userlevel 2
Badge +17

Hi @denby, I peeked the source HTML document of the page, and found this <form> element.

<form id="addressSearchForm" action="#addressSearchPanel" method="post" data-abide="ajax" class="panel">
  <div class="form-inline-label">
    <label for="cp4">Codigo Postal<span class="required">* </span></label>
  </div>
  <div class="medium-3 columns">
    <input type="text" name="cp4" maxlength="4" value="" id="cp4Field" data-msg-required="Campo obrigatorio" data-msg-pattern="Valor entre 1000 e 9999." data-rule-required="true" aria-required="" data-rule-pattern="^[1-9][0-9][0-9][0-9]$"/> 
  </div>
  <div class="medium-2 columns">
    <input type="text" name="cp3" maxlength="3" value="" id="cp3Field" data-msg-required="Campo obrigatorio" data-msg-pattern="Introduza 3 digitos." data-rule-required="true" aria-required="" data-rule-pattern="^[0-9][0-9][0-9]$"/> 
  </div>
  <div class="form-inline-buttons">
    <input id="addressSearchClear" type="button" class="button secondary" value="Limpar" />
    <button type="submit" id="button_search" name="method:searchPC2" value="Procurar">Procurar</button>
  </div>
</form>

So, the HTTPCaller with these parameters setting will get your desired response.

NameTypeValuecp4String Upload1000(for text form id = "cp4Field")cp3String Upload002(for text form id = "cp3Field")method:searchPC2String UploadProcurar(for submit button id = "button_search")

In the response body, the result appears as this <div> element.

<div class="highlighted-result text-left">
  <h4 class="subheader">Rua dos Açores</h4>
  <h4 class="subheader">Ímpares de 23 a 47</h4>
  <h3 class="subheader">Lisboa</h3>
  <h2>1000-002 LISBOA</h2>
</div>

You can extract the element from the response body with the HTMLExtractor.

0684Q00000ArKfHQAV.png

Then, it would be easy to parse the <div> element to extract individual items.

Hope this helps.

Userlevel 2
Badge +17

Hi @denby, I peeked the source HTML document of the page, and found this <form> element.

<form id="addressSearchForm" action="#addressSearchPanel" method="post" data-abide="ajax" class="panel">
  <div class="form-inline-label">
    <label for="cp4">Codigo Postal<span class="required">* </span></label>
  </div>
  <div class="medium-3 columns">
    <input type="text" name="cp4" maxlength="4" value="" id="cp4Field" data-msg-required="Campo obrigatorio" data-msg-pattern="Valor entre 1000 e 9999." data-rule-required="true" aria-required="" data-rule-pattern="^[1-9][0-9][0-9][0-9]$"/> 
  </div>
  <div class="medium-2 columns">
    <input type="text" name="cp3" maxlength="3" value="" id="cp3Field" data-msg-required="Campo obrigatorio" data-msg-pattern="Introduza 3 digitos." data-rule-required="true" aria-required="" data-rule-pattern="^[0-9][0-9][0-9]$"/> 
  </div>
  <div class="form-inline-buttons">
    <input id="addressSearchClear" type="button" class="button secondary" value="Limpar" />
    <button type="submit" id="button_search" name="method:searchPC2" value="Procurar">Procurar</button>
  </div>
</form>

So, the HTTPCaller with these parameters setting will get your desired response.

NameTypeValuecp4String Upload1000(for text form id = "cp4Field")cp3String Upload002(for text form id = "cp3Field")method:searchPC2String UploadProcurar(for submit button id = "button_search")

In the response body, the result appears as this <div> element.

<div class="highlighted-result text-left">
  <h4 class="subheader">Rua dos Açores</h4>
  <h4 class="subheader">Ímpares de 23 a 47</h4>
  <h3 class="subheader">Lisboa</h3>
  <h2>1000-002 LISBOA</h2>
</div>

You can extract the element from the response body with the HTMLExtractor.

0684Q00000ArKfHQAV.png

Then, it would be easy to parse the <div> element to extract individual items.

Hope this helps.

The HTTPCaller parameters dialog looks like this.

 

0684Q00000ArMWgQAN.png

Badge +16

I agree with @redgeographics if there is no API available the the only way is as @takashi proposed by posting to the site, but I wonder if the site is made for posting data into it.

Reply