Skip to main content
  • EN

Hello enthusiasts, 

I am trying to migrate a bulk data (assume 2Million records) to an oracle table due to size  planning to split and sent them by batches. In this situation I have a certain doubts which are below, 

  1. if a data already migrated to the oracle table, but next batch also have that record means oracle will through the “unique id constraint” error. how can I skip that record and move next one?
  2. Due to size of the data, i am splitting them into different batches and try to run parallel. FYI: I am migrating data into one same oracle table using oracle writer. can i run multiple workbenches (like insert1.fmw and insert2.fmw) parallelly and is it works fine or will be chances to loss data in this scenario ?

WRITER using: oracle spatial object 

expecting your thoughts and suggestions.

Thanks in advance !

The first issue could be tackled by doing an Upsert (not sure if Oracle supports that),or by checking for duplicates before writing.

I don’t think running multiple workspaces in parallel due to the size of the data is going to do you much good, you're still throwing all the data at the database in one go

Upsert can indeed be the way to go, but I am not sure if all Oracle variations support it. There is some upsert Oracle documentation out there: https://docs.oracle.com/en/database/other-databases/nosql-database/23.3/sqlreferencefornosql/upsert-statement.html#GUID-ED613A52-8A3B-481E-BF86-BB5FE1C29A5B

You can use an SQL statement (under ‘Advanced’ in the Oracle Spatial Object FeatureWriter) to implement the upsert, or to implement a check if your unique value is already present in the table before inserting it. 

Reply


Community Stats

30,351
Posts
115,341
Replies
38,081
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.