I am using Azure data factory(ADF) Pipeline-Copy Activity to copy data from cosmos db to datalake.I have created linkedservice for source (Cosmos db ) and destination (Data lake gen 2- Delimited format (.tsv))as I want my output in tsv format or csv.. which is working fine for me for simple objects. But,I want some suggestion for copying complex objects from cosmos db to flattened tsv format. eg:-> Source :-> { "id":"30b00315-bae9343442333", "TestId":"12345", "Information": [ {"Info": [ {"InfoID":1,"result":"4"}, {"InfoID":2,"result":"3"}, {"InfoID":3,"result":"3"}, {"InfoID":4,"result":"4"}, ] "lastModified":"2021-01-2T08:36:09.4961916", "lastModifiedBy":"SYSTEM", "flag":1 } ], "created":"2021-01-12T08:36:09.49561916", "createdBy":"SYSTEM", "isNew":true, "eTag":"\"6100478c-0000-0700-342343440000\"" } Destination should be in:-> Id TestId InfoId Result LastModified LastModifiedBy CreatedBy flag 30b00315-bae9343442333 12345 1 4 2021-01-2 System 2021-01-2 1 30b00315-bae9343442333 12345 2 3 2021-01-2 System 2021-01-2 1 30b00315-bae9343442333 12345 3 3 2021-01-2 System 2021-01-2 1 30b00315-bae9343442333 12345 4 4 2021-01-2 System 2021-01-2 1 As you can see above in source,Information is array and inside it,there is another field Info which is another array.I want this complex nested arrays in flattened structure. How we can achieve it? Please suggest some way. Thanks in advance.
1.Copy Activity Mapping schema :- But it will take only first index. 2.I have also tried for Data flow activity But it wont work for nested array objects.Let me know if i am missing something from my end.
var
This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)