Skip to main content

I'm making a standardized excel sheet that can easily transform my incoming data/features into data/features that i want in my own style. Transforming code into text, dates or numbers depending on the output style row in my translation table.

I have the following excel table style that i use for the translation:

category_old value_oldcategory_newsort_of_datavalue_newnote_about how this should worldatedate_of_constructiondatedates are just copiedroad_tpeasphaltroad_typeexisting_list01This list is already known in my databaseroad_tpeconcreteroad_typeexisting_list02This list is already known in my databaseroad_teelementsroad_typeexisting_list03This list is already known in my databasedmgAdamage_asphaltsplit01if road == asphalt fill in same valuedmgBdamage_asphaltsplit02if road == asphalt fill in same valuedmgAdamage_concretesplit01if road == concrete fill in same valuenamestreet_namenew_listMake a table with all uniqeu streets and count them 1 till x and add the matching number to the table

and the following attribute table for features:

IDdateroad_tpedmgname111-11-2016elementsA street211-11-2016asphaltBA street311-11-2016asphaltAB street411-11-2016concreteAC street

The result should look like

 

IDdate_of_constructionroad_typedamage_asphaltdamage_concretestreet_name111-11-201603nullnull01211-11-20160102null01311-11-20160101null02411-11-201602null0103

I currently write the SQL code to join the data in the InlineQuerier and add all the transformers manually after this, but its a lot of work with giant tables. I'm trying to make this completely automatic but getting stuck. The input should be a customized translation table for the dataset naming al the criteria and the output should always be a shp and in this case also an extra translation table for the street names (ID, street_nameein text] and code). Is it possible to make the information in the InlineQuerier adaptive to a list (like UniqueValueLogger)

This sounds like something the SchemaMapper might be able to tackle. Considering you already have your mapping table in Excel it might be relatively easy to set up. That page I linked to has some tutorials as well as use cases.

Hope this helps.

 


Hi @jdh009, I think that the InlineQuerier does the trick if you add a small preprocess. e.g.

  1. AttributeRenamer: Rename 'date' to 'date_of_construction'.

  2. DuplicateFilter: Extract unique street names.

  3. Counter: Generate 1-based sequential number as street names.

  4. StringFormatter: Format the street names with required format.

0684Q00000ArJpEQAV.png

Then, execute a SQL statement by the InlineQuerier with this setting.

0684Q00000ArKWZQA3.png

select
a.*,
b.value_new as road_type,
case
    when a.road_tpe = 'asphalt' then c.value_new
    else null
end as damage_asphalt,
case
    when a.road_tpe = 'concrete' then c.value_new
    else null
end as damage_concrete,
d.street_name as street_name
from Feature as a
inner join (select value_old, value_new
    from Style where category_old = 'road_tpe') as b
    on b.value_old = a.road_tpe
left outer join (select value_old, value_new, category_new
    from Style where category_old = 'dmg') as c
    on c.value_old = a.dmg and c.category_new like '%'||a.road_tpe
inner join Street as d on d.name = a.name
order by a.ID

I Agree with @redgeographics this is a job for the schema mapper.


Thank you for the answers and i'm looking into the schemamapper at the moment. It looks like a good option but it looks like you still need to name the attributes in the filters by name. 
I was wondering if i could make it more dynamic like this and based on values already given in the excel:

First you get a list of all the category_old unique features to start a loop and check what sort of data it is in Sort_of_data. 

According to what data it is you do the following actions.

for date_of_construction this would be:

If sort_of_date == date:
 string(category_newt1]) = date ##meaning: date_of_construction takes over the values of date

for road_tpe this should be something like:

If sort_of_date == existing_list:
*** make list of unique values based on the category_old and Value_old 
for i in (length list)
if value_old of road_tpe == value_old_listui]
value_old = value_new

if sort_of_date == split:
 split that takes road_tpe into account (road_tpe should be stored in a extra attributes split_category and split_value)
 after that its the same as above

if sort_of_date == new_list:
makes a list of all unique values and makes a new table of it that will be exported (name, code, order)
the code is just a count transformer
order is the list in alphabetical order
after this all street names should be replaced by the code.


Reply