Partially Uploaded Files

The SFTP protocol allows files to be partially uploaded. It provides the capability for data to be appended to the partial file, in anticipation of the rest of the file being uploaded later.

Unfortunately, an SFTP server is unaware of whether an uploaded file is complete or partial. Only the client app knows if there is remaining data to be uploaded later. To make matters worse, there's no way for a server to know what "later" means for a client app. The remaining data might be sent a few minutes, hours, or days later or never at all.

Files.com provides a site setting, Preserve Partial SFTP Uploads After Disconnects, that gives you control over how our SFTP service handles incomplete uploads. Depending on your requirements, disabling this setting may be the simplest way to prevent partial files from appearing on your site.

Some SFTP client apps implement an approach that uses a temporary file to upload into and then renames the temporary file to its target name. For example, when uploading file.csv a client will upload into a file named something like file.csv.filepart and then, after the complete file contents have been uploaded, rename file.csv.filepart back to file.csv.

SFTP client apps will only be able to resume the upload if the partial file remains at its location. Moving, renaming, or processing a partial file will remove the ability for the SFTP client app to resume the upload.

Some SFTP clients send SSH_FXP_CLOSE even when an upload is aborted or interrupted, meaning Files.com will treat a partial file as complete. This behavior is client-dependent and cannot be detected server-side.

To avoid partially uploaded files, use a Files.com native app to perform atomic uploads instead.

If your counterparty is limited to using SFTP and resume support is not a requirement, consider disabling the Preserve Partial SFTP Uploads After Disconnects setting, which will cause Files.com to automatically discard incomplete uploads when a client disconnects without explicitly closing the file.

If resume support is required, partial files cannot be avoided entirely, but their impact can be mitigated. Do not implement Automations, encryption, or Folder Settings on destination folders of SFTP uploads as these will act upon partial files. Try and target file name patterns that omit partial, or temporary, file names. Use schedules to delay actions on uploaded files. For example, if you expect files to be uploaded at 9 am then use an automation with a custom schedule set for 10 am, so that you allow an extra hour for any remaining data to be uploaded.