我有一个数据工厂 V2 管道,由“获取元数据”和“forEach”活动组成,用于读取文件共享(本地)上的文件列表并将其记录在数据库表中。目前,我只能读取文件名,但还想检索每个文件的修改日期和/或创建日期属性。有什么帮助吗?
谢谢
According to the MS documentation https://learn.microsoft.com/azure/data-factory/control-flow-get-metadata-activity.
We can see File system
and SFTP
both support the lastModified property. But we only can get the lastModified of one file or folder at a time.
I'm using File system
to do the test. The process is basically the same as the previous post https://stackoverflow.com/questions/65161230/azure-data-factory-v2-copy-activity-save-list-of-all-copied-files, we need to add a GetMetaData
activity to the ForEach activity.
This is my local files.
- 首先,我创建了一个用于记录的表。
create table Copy_Logs (
Copy_File_Name varchar(max),
Last_modified datetime
)
-
In ADF, I'm using Child Items
at Get Metadata1
activity to get the file list of the folder.
-
Then add dynamic content @activity('Get Metadata1').output.childItems
at ForEach1
activity.
-
Inside the ForEach1
activity, using Last modified
at Get Metadata2
activity.
In the dataset of Get Metadata2
activity, I key in @item().name
as follows.
-
Using CopyFiles_To_Azure
activity to copy local files to the Azure Data Lake Storage V2.
I key in @item().name
at the source dataset of CopyFiles_To_Azure
activity.
-
At Create_Logs
activity, I'm using the following sql to get the info we need.
select '@{item().name}' as Copy_File_Name, '@{activity('Get Metadata2').output.lastModified}' as Last_modified
- In the end, sink to the sql table we created previously. The result is as follows.
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)