Question

limit CSV writer size

  • 13 October 2014
  • 2 replies
  • 5 views

Badge +1
I have a CSV output file which is currently 54mb. 

 

I need to upload it in a system which has a limit of 5mb. 

 

 

Is there a way of limiting the writer to 5mb limit and then write to another CSV. 

 

 

i.e. I want to get 11 csv's of the same data. Ideally with a counter in the name i.e. upload1, upload2, etc.

 

 

Is there a way of either setting a writter parameter or a transformer to do this? 

 

 

 


2 replies

Userlevel 4
Hi,

 

 

one way to do it:
  • find out approximately how many lines it takes to get 5MB
  • use a ModuloCounter set to this number (or slightly lower to be conservative)
  • use _modulo_count as a basis for a feature fanout

 

David
Badge +1
Many thanks. 

Reply