Prev
Next
Forums Integration / Customization IPA/ProcessFlow

combining data from 2 csv files

 3 Replies
 3 Subscribed to this topic
 52 Subscribed to this forum
Sort:
Author
Messages
adnan512
Advanced Member
Posts: 24
Advanced Member
1/26/2016 2:49 AM

    Hello all.

    I am trying to read two csv files and output a 3rd csv file with fields from fileA and fileB. Both files have a common key (ad username). 

    I am reading FileA through data iterator node storing the key field and I have a 2nd data iterator node to read the 2nd file to find the records based on the key filed. I am running into a problem when next record is read... data iterator for 2nd file does not start from the beginning instead it reads the following records from the previous loop. I have tried to set the next position to 0 and recordnumber to 0 with no success. I am trying to avoid using an array. Any help or idea would be appreciated.

    Thanks

    Shane Jones
    Veteran Member
    Posts: 460
    Veteran Member
    1/26/2016 4:29 AM
      It might work if you set the data in the 2nd file as a variable before doing iteration. Then you could empty the value at the end of the 1st iteration and re-read it into the variable.
      Shane Jones
      Tools: HR, Payroll, Benefits, PFI, Smart Office, BSI, Portal and Self-Service
      Systems: Lawson, Open Hire, Kronos, Crystal Reporting, SumTotal Learning
      ** Teach others to fish...
      David Williams
      Veteran Member
      Posts: 1127
      Veteran Member
      1/26/2016 11:30 AM
        Instead of using the DataIterator to read the file, load the file contents with a FileAccess and interate the Data (FilleAccess_output) loaded. This would actually be faster.
        I'm curious, however, what would be the problem of using arrays? You can create an array variable without defining it as an array.

        I've done this type of activity many times, loading the files into memory (assigning the output to variables) and then using JavaScript (within an Assign) to compare the files and output a new set of data (which could then be written to your 3rd file).
        David Williams
        Terry P
        Veteran Member
        Posts: 234
        Veteran Member
        1/26/2016 3:07 PM
          Do the records need to be merged, or just combined? If the latter, then try doing the system command of "cat" might work. Example: cat file1 file2 >file3