Question

standardized translation table and dynamic query

  • 11 November 2016
  • 4 replies
  • 1 view

Badge +2

I'm making a standardized excel sheet that can easily transform my incoming data/features into data/features that i want in my own style. Transforming code into text, dates or numbers depending on the output style row in my translation table.

I have the following excel table style that i use for the translation:

category_old value_oldcategory_newsort_of_datavalue_newnote_about how this should worldatedate_of_constructiondatedates are just copiedroad_tpeasphaltroad_typeexisting_list01This list is already known in my databaseroad_tpeconcreteroad_typeexisting_list02This list is already known in my databaseroad_teelementsroad_typeexisting_list03This list is already known in my databasedmgAdamage_asphaltsplit01if road == asphalt fill in same valuedmgBdamage_asphaltsplit02if road == asphalt fill in same valuedmgAdamage_concretesplit01if road == concrete fill in same valuenamestreet_namenew_listMake a table with all uniqeu streets and count them 1 till x and add the matching number to the table

and the following attribute table for features:

IDdateroad_tpedmgname111-11-2016elementsA street211-11-2016asphaltBA street311-11-2016asphaltAB street411-11-2016concreteAC street

The result should look like

 

IDdate_of_constructionroad_typedamage_asphaltdamage_concretestreet_name111-11-201603nullnull01211-11-20160102null01311-11-20160101null02411-11-201602null0103

I currently write the SQL code to join the data in the InlineQuerier and add all the transformers manually after this, but its a lot of work with giant tables. I'm trying to make this completely automatic but getting stuck. The input should be a customized translation table for the dataset naming al the criteria and the output should always be a shp and in this case also an extra translation table for the street names (ID, street_name[in text] and code). Is it possible to make the information in the InlineQuerier adaptive to a list (like UniqueValueLogger)


4 replies

Userlevel 5
Badge +25

This sounds like something the SchemaMapper might be able to tackle. Considering you already have your mapping table in Excel it might be relatively easy to set up. That page I linked to has some tutorials as well as use cases.

Hope this helps.

 

Userlevel 2
Badge +17

Hi @jdh009, I think that the InlineQuerier does the trick if you add a small preprocess. e.g.

  1. AttributeRenamer: Rename 'date' to 'date_of_construction'.

  2. DuplicateFilter: Extract unique street names.

  3. Counter: Generate 1-based sequential number as street names.

  4. StringFormatter: Format the street names with required format.

0684Q00000ArJpEQAV.png

Then, execute a SQL statement by the InlineQuerier with this setting.

0684Q00000ArKWZQA3.png

select
a.*,
b.value_new as road_type,
case
    when a.road_tpe = 'asphalt' then c.value_new
    else null
end as damage_asphalt,
case
    when a.road_tpe = 'concrete' then c.value_new
    else null
end as damage_concrete,
d.street_name as street_name
from Feature as a
inner join (select value_old, value_new
    from Style where category_old = 'road_tpe') as b
    on b.value_old = a.road_tpe
left outer join (select value_old, value_new, category_new
    from Style where category_old = 'dmg') as c
    on c.value_old = a.dmg and c.category_new like '%'||a.road_tpe
inner join Street as d on d.name = a.name
order by a.ID
Badge +16

I Agree with @redgeographics this is a job for the schema mapper.

Badge +2

Thank you for the answers and i'm looking into the schemamapper at the moment. It looks like a good option but it looks like you still need to name the attributes in the filters by name. 
I was wondering if i could make it more dynamic like this and based on values already given in the excel:

First you get a list of all the category_old unique features to start a loop and check what sort of data it is in Sort_of_data. 

According to what data it is you do the following actions.

for date_of_construction this would be:

If sort_of_date == date:
 string(category_new[1]) = date ##meaning: date_of_construction takes over the values of date

for road_tpe this should be something like:

If sort_of_date == existing_list:
*** make list of unique values based on the category_old and Value_old 
for i in (length list)
if value_old of road_tpe == value_old_list[i]
value_old = value_new

if sort_of_date == split:
 split that takes road_tpe into account (road_tpe should be stored in a extra attributes split_category and split_value)
 after that its the same as above

if sort_of_date == new_list:
makes a list of all unique values and makes a new table of it that will be exported (name, code, order)
the code is just a count transformer
order is the list in alphabetical order
after this all street names should be replaced by the code.

Reply