Solved

How to read an ArcGIS feature layer, join two tables, and write to a another ArcGIS feature layer?

datablue
Rank
Badge +7

I'm using an ArcGIS Feature reader to read a layer with two tables. The two tables have 9257 and 9606 rows. After reading the two tables, I join them. At this point (with feature caching) I can see that I have 9606 rows. This is correct. The last step of the workspace is to use an ArcGIS Feature writer to populate a different layer. 

 

The ArcGIS Feature Writer has the following Parameters for Layer:

 Writer Mode: Insert

 Truncate First: Yes

 

Currently, my destination Feature layer has 10584 rows. How could this be? How can I exceed the number of rows in my source layer? I just ran the workspace up to the Writer and can see 9606 items going into the Feature Writer. then, I ran the entire workspace and the destination layer now has 9752 rows. I'm perplexed. This is a public facing dashboard and I need accuracy.

 

I do run this workspace from a batch file and schedule it to run at 3:00 AM with Windows Task Scheduler. I'm not sure if that matters.

 

Any ideas? I suspect that the destination layer is not being truncated before inserting new rows.

 

Update: Two minutes after posting, I refreshed the web page showing data for the destination layer and now I have 11606 rows! I must be doing something wrong.

icon

Best answer by ctredinnick 10 May 2022, 05:51

View original

3 replies

ctredinnick
Rank
Userlevel 3
Badge +16

Is there another process writing into the same feature layer? It seems like 9606 features were written by FME, then every time you checked subsequently the feature count had increased.

You can double check at the end of the log file after FME has finished, there will be a summary of features written.

datablue
Rank
Badge +7

Is there another process writing into the same feature layer? It seems like 9606 features were written by FME, then every time you checked subsequently the feature count had increased. 

You can double check at the end of the log file after FME has finished, there will be a summary of features written.

Thanks for your suggestion. I was not able to check the log file yesterday and I'm not sure if I can access a log file of a previously run workspace. This morning at 3am the workspace was run from the batch file and now there are 6000 records.

 

I checked the log file discovered that after 6000 features were written there was this error:

2022-05-10 04:02:14|  28.3|  0.3|ERROR |ArcGIS Online Feature Service Writer: Could not connect to server. Check that you are connected to the internet and that no firewall is blocking outbound connections. The error was 'HTTPSConnectionPool(host='services7.arcgis.com', port=443): Read timed out.'

Near the end of the log file it said:

Total Features Written: 6000

Translation FAILED with 2 error(s) and 295 warning(s) (9609 feature(s) output)

 

datablue
Rank
Badge +7

Thanks for your suggestion. I was not able to check the log file yesterday and I'm not sure if I can access a log file of a previously run workspace. This morning at 3am the workspace was run from the batch file and now there are 6000 records.

 

I checked the log file discovered that after 6000 features were written there was this error:

2022-05-10 04:02:14|  28.3|  0.3|ERROR |ArcGIS Online Feature Service Writer: Could not connect to server. Check that you are connected to the internet and that no firewall is blocking outbound connections. The error was 'HTTPSConnectionPool(host='services7.arcgis.com', port=443): Read timed out.'

Near the end of the log file it said:

Total Features Written: 6000

Translation FAILED with 2 error(s) and 295 warning(s) (9609 feature(s) output)

 

I manually ran it again. This time there was no error connecting to the server, but it wrote about 1000 more features than expected. I started a chat session with FME Support.  

 

After I re-installed 2021.2.3 and ran the workspace again (and downgraded the FeatureJoiner), the issue went away. I don't know if this is a bug in 2022.0 or if I misconfigured the new FeatureJoiner. Either way, a case has been created. Thanks.

Reply


Community Stats

28,723
Posts
109,510
Replies
36,559
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.