Exporting SQL Server Table to Multiple Part Files

How to export from sql server table to multiple csv files in Azure Data Factory

You would need to follow the below flow chart:

1. LookUp Activity : Query : Select distinct city from table
2. For each activity
   
   Input : @activity('LookUp').output.value
   a) Copy activity
   i) Source : Dynamic Query Select * from t1 where city=@item().City

This should generate separate files for each country as needed

Steps:
1)

2. The batch job can be any nbr of parallel executions
   

4. Create a parameterised dataset:
   

5)

Result: I have 2 different Entities, so 2 files are generated.

Input :

Output:

Inserting huge batch of data from multiple csv files into distinct tables with Postgresql

If you use DBeaver, there is a recently-added feature in the software which fixes this exact issue. (On Windows) You have to right click the section "Tables" inside your schemas (not your target table!) and then just select "Import data" and you can select all the .csv files you want at the same time, creating a new table for each file as you mentioned.

Related Topics