Skip to main content
Solved

Create date segments of two table with overlap of times


kevinkies
Contributor
Forum|alt.badge.img+3

Hello,

I have two tables with the same field names. Both tables have 2 date fields, with a start date and a end date and other fields which has the same name in both files. The only thing is that the date segmenst (time between start and end) has a overlap.

Example:

TABLE 1:

DATUM_INGANG DATUM_EINDE ID REPLACER
19921101 20040119 0502010000027183 OWNER
20040119 20160930 0502010000027183 OWNER
20160930 99991231 0502010000027183 OWNER

 

TABLE 2:

 

DATE_START DATE_END ID REPLACER
19921101 20060716 0502010000027183 USER
20060716 20061008 0502010000027183 USER
20061008 20070220 0502010000027183 USER
20070220 20100616 0502010000027183 USER
20100616 20110709 0502010000027183 USER
20110709 20111001 0502010000027183 USER
20111001 20190325 0502010000027183 USER
20190325 20190617 0502010000027183 USER
20190617 99991231 0502010000027183 USER

 

I want to combine them to one table, this should be the outcome:

 

DATE_START DATE_END ID REPLACER
19921101 20040119 0502010000027183 EIGENAAR
20040119 20060716 0502010000027183 EIGENAAR
20060716 20061008 0502010000027183 GEBRUIKER
20061008 20070220 0502010000027183 GEBRUIKER
20070220 20100616 0502010000027183 GEBRUIKER
20100616 20110709 0502010000027183 GEBRUIKER
20110709 20111001 0502010000027183 GEBRUIKER
20111001 20160930 0502010000027183 GEBRUIKER
20160930 20190325 0502010000027183 EIGENAAR
20190325 20190617 0502010000027183 GEBRUIKER
20190617 99991231 0502010000027183 GEBRUIKER

 

The date_end 99991231 is for tohave a value in the future. U can see that the dates have been segments. I trying it in FME, with listbuilders, listindexers etc. But it wont work, can someone help me?

 

In the attachement the real table. I want to do this with a large data amount, one the ID is must me grouped.

Best answer by takashi

I updated the workflow according to the rules.

 

View original
Did this help you find an answer to your question?

25 replies

takashi
Evangelist
  • July 9, 2025

Hi ​@kevinkies ,

If you consider each time duration as a line segment from (date start, 0) to (date end, 0), you can find overlapped segments with LineOnLineOverlayer easily.

Basic workflow would look like the screenshot below. Hovever, I'm not clear detailed requirement about how to keep which value of LEEGSTAND and other attributes from Table 1 or Table 2 in overlaped segments, how to sort resulting rows and so on, so you will have to customize the workflow according to your requirement.

 


ebygomm
Influencer
Forum|alt.badge.img+39
  • Influencer
  • July 9, 2025

You could do it spatially, convert dates to seconds since epoch, use that to create boxes, spatially intersect those with an areaonareaoverlayer, get the min and max coordinates which are the start and end dates of each segment in seconds and convert that back to a date

 

 


kevinkies
Contributor
Forum|alt.badge.img+3
  • Author
  • Contributor
  • July 10, 2025

Hi ​@takashi , it works but indeed with no attribute so for. And i need them.

 

So i tried the solution of ​@ebygomm it works so far, but there is one thing that isnt good. That is if u use the table in the attachmenst. This is my outcome:

 

DATE_START DATE_END ID BEDRIJFSNAAM LEEGSTAND EIGDOM_SITUATIE WIJZIGING
19921101 20040119 0502010000027183 Van der Vorm Vastgoed Holding B.V. N Particuliere verhuur GEBRUIKER
20040119 20060716 0502010000027183 Van der Vorm Vastgoed Holding B.V. N Particuliere verhuur GEBRUIKER
20060716 20061008 0502010000027183 Van der Vorm Vastgoed B.V. N Particuliere verhuur GEBRUIKER
20061008 20070220 0502010000027183 Van der Vorm Vastgoed B.V. J   GEBRUIKER
20070220 20100616 0502010000027183 Van der Vorm Vastgoed B.V. N Particuliere verhuur GEBRUIKER
20100616 20110709 0502010000027183 Van der Vorm Vastgoed B.V. N Particuliere verhuur GEBRUIKER
20110709 20111001 0502010000027183 Van der Vorm Vastgoed B.V. J   GEBRUIKER
20111001 20160930 0502010000027183 Van der Vorm Vastgoed B.V. N Particuliere verhuur GEBRUIKER
20160930 20190325 0502010000027183 Van der Vorm Vastgoed B.V. N Particuliere verhuur GEBRUIKER
20190325 20190617 0502010000027183 Woonhave Beleggingen B.V. J   GEBRUIKER
20190617 99991231 0502010000027183 Woonhave Beleggingen B.V. N Particuliere verhuur GEBRUIKER

 

There are only 2  misstake in the table, BEDRIJFSNAAM is wrong its start one row to early or to late and WIJZIGING is ever row the samen. Further its perfect thanks!

 

Can u help me with that?

 


ebygomm
Influencer
Forum|alt.badge.img+39
  • Influencer
  • July 10, 2025

Can you explain the logic for how the attributes should be assigned? I couldn’t work it out


kevinkies
Contributor
Forum|alt.badge.img+3
  • Author
  • Contributor
  • July 10, 2025

The first two rows (previous post) has the value in BEDRIJFSNAAM = Van der Vorm Vastgoed Holding B.V.

Only if u see in table_1 that the value of BEDRIJFSNAAM= Van der Vorm Vastgoed Holding B.V. ends on the date 20040119:

DATE_START DATE_END ID BEDRIJFSNAAM
19921101 20040119 0502010000027183 Van der Vorm Vastgoed Holding B.V.

 

So bewteen 20040119 till 20160930 must be the value of BEDRIJFSNAAM = Van der Vorm Vastgoed B.V. like in table_1 is.

 

But in the outcome in my previous post is the value of BEDRIJFSNAAM=an der Vorm Vastgoed Holding B.V. ends on 20060716, and thats incorrect


ebygomm
Influencer
Forum|alt.badge.img+39
  • Influencer
  • July 10, 2025

But in Table2 there is record that goes beyond 20040119 which also has a value of Van der Vorm Vastgoed Holding B.V. ?

 

In the AreaOnAreaOverlayer, you can choose to drop the attributes in Accumulation Mode and then build a List with the attributes and then use some logic to choose which value of BEDRIFJSNAAM you want to keep (I’m afraid i still don’t understand how you arrive at this so can’t help there!)

 


kevinkies
Contributor
Forum|alt.badge.img+3
  • Author
  • Contributor
  • July 10, 2025

Them u are right! I only want the use de BEDRIJFSNAAM of table_1:

DATE_START DATE_END ID BEDRIJFSNAAM
19921101 20040119 0502010000027183 Van der Vorm Vastgoed Holding B.V.
20040119 20160930 0502010000027183 Van der Vorm Vastgoed B.V.
20160930 99991231 0502010000027183 Woonhave Beleggingen B.V.

 

So he must use the value betweens the date_start and date_end at the end. The BEDRIJFSNAAM in table_2 can be blank then.

 

Is that possible u think. If i make the values empty in table_2, then the outcome has no values filled.


takashi
Evangelist
  • July 10, 2025

@kevinkies 

it works but indeed with no attribute so for. And i need them.

Overlapped segments will have two attribute sets from Table 1 and Table 2, and they all will be stored in a list attribute in my workflow example. I don't know which one should be kept finally, so the workflow is unfinished, as I told you that you will have to customize the workflow.

What are the rules for transferring the source two attribute sets to overlapped segments?


ebygomm
Influencer
Forum|alt.badge.img+39
  • Influencer
  • July 10, 2025

If you want the values from Table 1 to take precedence I think if you add an attribute to your input, e.g. sort and give a value of 1 for Table 1 and 2 for Table 2, you can then include this in the list you build in the area on area overlayer, sort the list on this value and then set BEDRIFJSNAAM like this

 

 


kevinkies
Contributor
Forum|alt.badge.img+3
  • Author
  • Contributor
  • July 10, 2025

@takashi the rules are as follows:

  • date_start and date_end from bothe tables
  • ID is key to connect both tables
  • BEDRIJFSNAAM from table_1
  • LEEGSTAND from table_2
  • Eigdom_situatie from table_2
  • WIJZIGING from both tables (but not important)
    Is this enough?

takashi
Evangelist
  • Best Answer
  • July 10, 2025

I updated the workflow according to the rules.

 


kevinkies
Contributor
Forum|alt.badge.img+3
  • Author
  • Contributor
  • July 10, 2025

thanks ​@takashi it works fine. Only one comment left. I know i had write that the row WIJZIGING isnt very relevant. But since we are no so far, i realyy wont the have it perfect.

 

No the value of WIJZIGING is all the same name, but in reality it isnt. In 3 rows the value of WIJZIGING is EIGENAAR and the others are GEBRUIKER.

The rows who must have EIGENAAR are:

DATE_START DATE_END WIJZIGING
19921101 20040119 EIGENAAR
20040119 20060716 EIGENAAR
20160930 20190325 EIGENAAR

 

Is that also possible to fix?


takashi
Evangelist
  • July 10, 2025

I think it's possible, if you would clarify how you can determine which "WIJZIGING" value - from table 1 or table 2 should be assigned to the resulting segment.


takashi
Evangelist
  • July 11, 2025

In my observasion, it seems that the condition to adopt WIJZIGING from the table 1 in the resulting rows is, DATE_START in the resulting segment is equal the DATE_START from the table 1. If so, this workflow might help you.

 


kevinkies
Contributor
Forum|alt.badge.img+3
  • Author
  • Contributor
  • July 11, 2025

@takashi thanks. I add the same rules and also did it for table_2 the same way. So i have 3 possible values in de manager add WIJZIGING. Now he checked al possible start datum with the good value. 

I appreciate your help!

 

One question left :-)

 

Is it possible to copy the rows to a specific date, i mean as follows:

DATE_START DATE_END ID BEDRIJFSNAAM LEEGSTAND EIGDOM_SITUATIE WIJZIGING
19921101 20040119 0502010000027183 Van der Vorm Vastgoed Holding B.V. N Particuliere verhuur EIGENAAR

 

TO:

DATE ID BEDRIJFSNAAM LEEGSTAND EIGDOM_SITUATIE WIJZIGING
19921101 0502010000027183 Van der Vorm Vastgoed Holding B.V. N Particuliere verhuur EIGENAAR
19921102 0502010000027183 Van der Vorm Vastgoed Holding B.V. N Particuliere verhuur EIGENAAR
19921103 0502010000027183 Van der Vorm Vastgoed Holding B.V. N Particuliere verhuur EIGENAAR
…... .. ..
20040119 0502010000027183 Van der Vorm Vastgoed B.V. N Particuliere verhuur EIGENAAR
20040120 0502010000027183 Van der Vorm Vastgoed B.V. N Particuliere verhuur EIGENAAR
20040121 0502010000027183 Van der Vorm Vastgoed B.V. N Particuliere verhuur EIGENAAR
….. .. ..
20060716 0502010000027183 Van der Vorm Vastgoed B.V. N Particuliere verhuur GEBRUIKER
20060717 0502010000027183 Van der Vorm Vastgoed B.V. N Particuliere verhuur GEBRUIKER
20060718 0502010000027183 Van der Vorm Vastgoed B.V. N Particuliere verhuur GEBRUIKER
….. .. ..

 

What i mean, now ik have date segment, and want to tranfer the segment to all days betweens the segment. The dots means the dates between.


takashi
Evangelist
  • July 11, 2025

Yes, you can do that with DateTimeCalculator x 2 + Cloner.

However, at first, you have to define how the DATE_END 99991231 should be treated. The number of days from 20190617 to 99991231 is approx. 2.9 millions, do you want to copy such a large number of rows?


kevinkies
Contributor
Forum|alt.badge.img+3
  • Author
  • Contributor
  • July 11, 2025

99991231 is a value to de de math in the first steps. It is actually a null value, because that row actually has no date_end. The object is still in use. It is current day


takashi
Evangelist
  • July 11, 2025

Do you mean that the DATE_END 99991231 should be treated as the date of today - i.e. 2025-07-11 in my time zone when copying the rows?


kevinkies
Contributor
Forum|alt.badge.img+3
  • Author
  • Contributor
  • July 11, 2025

Yes indeed!


takashi
Evangelist
  • July 11, 2025

Try adding these transformers to the previous workflow. Hope this helps!

 


kevinkies
Contributor
Forum|alt.badge.img+3
  • Author
  • Contributor
  • July 11, 2025

@takashi thanks, everything works


kevinkies
Contributor
Forum|alt.badge.img+3
  • Author
  • Contributor
  • July 15, 2025

@takashi there where 159 million records, myabe to big for oracle. Is it posiible to change it to only the first day of the month instead of every day?

 

Thanks already


takashi
Evangelist
  • July 15, 2025

Do you need to retain the date equal to the DATE_START for each segment?

 


kevinkies
Contributor
Forum|alt.badge.img+3
  • Author
  • Contributor
  • July 16, 2025

Yes indeed. Now i see thats a problem if its nog begin on the first of the month probaly?


takashi
Evangelist
  • July 16, 2025

Try replacing the transformers from Sorter to DateTimeCalulator_2 in the previous workflow with these transformers. A bit complex, take a look at details carefully.

 


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings