Skip to main content
Solved

How to read an ArcGIS feature layer, join two tables, and write to a another ArcGIS feature layer?

datablue
Contributor
Forum|alt.badge.img+7

I'm using an ArcGIS Feature reader to read a layer with two tables. The two tables have 9257 and 9606 rows. After reading the two tables, I join them. At this point (with feature caching) I can see that I have 9606 rows. This is correct. The last step of the workspace is to use an ArcGIS Feature writer to populate a different layer. 

 

The ArcGIS Feature Writer has the following Parameters for Layer:

 Writer Mode: Insert

 Truncate First: Yes

 

Currently, my destination Feature layer has 10584 rows. How could this be? How can I exceed the number of rows in my source layer? I just ran the workspace up to the Writer and can see 9606 items going into the Feature Writer. then, I ran the entire workspace and the destination layer now has 9752 rows. I'm perplexed. This is a public facing dashboard and I need accuracy.

 

I do run this workspace from a batch file and schedule it to run at 3:00 AM with Windows Task Scheduler. I'm not sure if that matters.

 

Any ideas? I suspect that the destination layer is not being truncated before inserting new rows.

 

Update: Two minutes after posting, I refreshed the web page showing data for the destination layer and now I have 11606 rows! I must be doing something wrong.

Best answer by ctredinnick

Is there another process writing into the same feature layer? It seems like 9606 features were written by FME, then every time you checked subsequently the feature count had increased.

You can double check at the end of the log file after FME has finished, there will be a summary of features written.

View original
Did this help you find an answer to your question?

3 replies

ctredinnick
Supporter
Forum|alt.badge.img+18
  • ctredinnickSupporter
  • Supporter
  • Best Answer
  • May 10, 2022

Is there another process writing into the same feature layer? It seems like 9606 features were written by FME, then every time you checked subsequently the feature count had increased.

You can double check at the end of the log file after FME has finished, there will be a summary of features written.

datablue
Contributor
Forum|alt.badge.img+7
  • datablueContributor
  • Author
  • Contributor
  • May 10, 2022
ctredinnick wrote:

Is there another process writing into the same feature layer? It seems like 9606 features were written by FME, then every time you checked subsequently the feature count had increased. 

You can double check at the end of the log file after FME has finished, there will be a summary of features written.

Thanks for your suggestion. I was not able to check the log file yesterday and I'm not sure if I can access a log file of a previously run workspace. This morning at 3am the workspace was run from the batch file and now there are 6000 records.

 

I checked the log file discovered that after 6000 features were written there was this error:

2022-05-10 04:02:14|  28.3|  0.3|ERROR |ArcGIS Online Feature Service Writer: Could not connect to server. Check that you are connected to the internet and that no firewall is blocking outbound connections. The error was 'HTTPSConnectionPool(host='services7.arcgis.com', port=443): Read timed out.'

Near the end of the log file it said:

Total Features Written: 6000

Translation FAILED with 2 error(s) and 295 warning(s) (9609 feature(s) output)

 

datablue
Contributor
Forum|alt.badge.img+7
  • datablueContributor
  • Author
  • Contributor
  • May 10, 2022
datablue wrote:

Thanks for your suggestion. I was not able to check the log file yesterday and I'm not sure if I can access a log file of a previously run workspace. This morning at 3am the workspace was run from the batch file and now there are 6000 records.

 

I checked the log file discovered that after 6000 features were written there was this error:

2022-05-10 04:02:14|  28.3|  0.3|ERROR |ArcGIS Online Feature Service Writer: Could not connect to server. Check that you are connected to the internet and that no firewall is blocking outbound connections. The error was 'HTTPSConnectionPool(host='services7.arcgis.com', port=443): Read timed out.'

Near the end of the log file it said:

Total Features Written: 6000

Translation FAILED with 2 error(s) and 295 warning(s) (9609 feature(s) output)

 

I manually ran it again. This time there was no error connecting to the server, but it wrote about 1000 more features than expected. I started a chat session with FME Support.  

 

After I re-installed 2021.2.3 and ran the workspace again (and downgraded the FeatureJoiner), the issue went away. I don't know if this is a bug in 2022.0 or if I misconfigured the new FeatureJoiner. Either way, a case has been created. Thanks.

Reply


Most helpful members this week

Community Stats

30,872
Posts
117,040
Replies
38,692
Members
Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.