Maybe there is already a way to do this, but I haven’t found it yet. As I am sitting here, waiting for the only remaining single, large file to be synchronized on a 4 thread machine, I am thinking that it would be great to have a way to control the order of the synchronization. Admins could then select large files to start on some of the available threads at the beginning of the sync and process many small files on the other threads at the same time, instead of waiting to the middle or end of an alphabetical sync list to start a massive file sync.
I am thinking of a priority system, but any method of controlling the order of files to be synchronized would be helpful. The Groups option doesn’t seem very helpful when you have to sync 400+ files in one maintenance window as quickly as possible.
1 Like
Yeah, I get what you mean. Right now, AWS DataSync processes files in a mostly alphabetical order, and there’s no built-in way to prioritize large files upfront. One workaround could be splitting the sync job into multiple tasks—one for large files and another for smaller ones—so they run in parallel. You could also try using filters to exclude large files initially, sync the smaller ones first, and then run a second sync for the big ones. Not perfect, but it might help optimize the process.