Best Practice when reading a file into IPA

Hi Joan - I would think that using a fileAccess node first and then using the dataIterator against the fileAccess output data would result in better performance. The only thing to be concerned about is the size of the source file. By using the fileAccess node, you are loading the entire document into memory, so if you have memory constraints or issues on your IPA server then this might be a concern. However, if the file is not unreasonably huge, I'd recommend that approach.

I doubt that the ftp node would provide any help with this, because I think it would just be used to move the file between servers rather than exposing the file contents to the flow.

Kelly

Hi Woozy - thank you.  I'm assuming that it doesn't matter if the file is located on the same server as IPA, your answer would still be the same.

Joan

It might be this link:

https://www.lawsonguru.co...rator-login-process/

Hi Joan - In order to use either the fileAccess or dataIterator nodes to read a file, the file has to live on the IPA server, or at least be accessible to it via some sort of file share. This may also be possible by using a different configuration set, but I've never tried that. Otherwise you would need to use an ftp node (or some other file transfer mechanism) to get the file from its source location and move it to the IPA server before reading it.

tambrosi thanks for the link, that's the one I was looking for.

woozy - Thank you.  One more question.  We are using a File Channel to read in the FILE so the DI input method is DATA from _inputData. based on a post in the link below

 "The idea otf using FileAccess to 'load' the file into IPA prevents multipe Open-Read-Close of a file that the DataItera performs when it processes a file. Load the file and then set the DataIterator to process the Data (instead of File) and set the source to be the FileAccess_outputData. So read the file into IPA and set the DI to process the data from that. It saves a lot of time."

it appears the performance issue is related to the DI accessing the FILE as appose to the DATA.  So in this case a file access is not needed.  Is this a correct assumption?

Joan

Joan - yep, if you are using a file channel, then loading the _inputData into the dataIterator should do the trick.

Some cautions about using the file channel depending on how large the files are.
If you are using the channel/receiver option to read and pass the data, the input data becomes part of the work unit.
The method i would suggest is to pass in the name rather than the data.
Then in the flow, you can read the file (using either data iterator or fileaccess--no significant penalty since the file is on IPA server), and archive or delete it.
My point is that using that method will guarantee that you retain a copy of the file.

Thank you everyone, that's what I needed.

Joan