I have a job coming in that is 800k pages in total that is made up of several document types which need merging based on address, I’ve been able to chunk it down slightly so it’s not 800k in one hit. However I’ve just run the 1st chunk of 250k pages and it’s taken 13 hours to run the job. There is still another stage after this that needs doing as this stage sorted the data by address and the next stage will then split the documents up based on address and add production marks etc.
Looking at the logs it took 8 hours to do the actual output stage
This is obviously a very big worry as in total this is going to take days to process by the looks of it and we don’t have days to process everything as there is a large amount of days worth of printing and enclosing. Below is a screen shot of the output settings we are using, can anyone suggest anything that may speed up the processing.
Just as a FYI also, this is raw data that comes in so all PDF’s have been created by PReS. The steps it goes through is:
- Run several data files through relevant document type mapper to create PDF’s and split into relevant output files
- Run PDF’s from different document types in batches where possible through mapper etc and sort output by address
- Run sorted PDF’s back through and split based on address changing and add production marks etc
I’ve already split each of the above steps into different processes as we have been getting IPC connection errors so I don’t want to loose days worth of processing if it does fall over and we have to start from the beginning again.