goatlady
I'm trying to upload a large file to Amazon S3 using CloudFTP.

My understanding is that S3 allows files up to 5GB, so I created a multi-part archive with each part 4.3GB. The file uploads and then fails right at the end.

Smart FTP login file:
[22:36:18] MLST archivedmail.part1.rar
[22:36:19] 550 No such file or directory
[22:36:19] PASV
[22:36:19] 227 Entering Passive Mode (209,188,12,218,78,118).
[22:36:19] Opening data connection to 209.188.12.218 Port: 20086
[22:36:19] STOR archivedmail.part1.rar
[22:36:19] 150 File status okay; about to open data connection.
[06:01:40] 4698669056 bytes transferred. (171 KB/s) (07:25:20)
[06:02:40] Timeout (60s).
[06:02:40] Active Help: http://www.smartftp.com/support/kb/74
[06:02:40] Client closed the connection.
[06:02:40] Transfer failed.
[06:02:40] Operation End

This has happened three times now on the same file. Does CloudFTP have a smaller maximum file size? I've uploaded approximately 2500 files between 15MB and 100MB each already without problem.
Quote
admin
I think you will find that the file was actually uploaded and does actually reside on S3. I see from the logs that the FTP client timed out but the file continued to be uploaded and completed. You should be able to verify this.
Quote
goatlady

SmartFTP attempts to re-upload the file from the beginning - as it takes quite some time to upload 5GB I haven't caught it in the act, but I assume it's getting to the end, is unable to verify the file, and restarts. I have checked on it and found for example that while I last saw it at 60%, several hours later it was at 20%. I've stopped it now so the file that resides on S3 is not complete. This is not normal behaviour - in my use of SmartFTP it's common for FTP connections to time out but the transfer continues in the background successfully.

I will try breaking the files into smaller pieces and see how that works.  

Quote
goatlady

I tried with a 700mb file and when it finished uploading, the log shows 

[12:44:04] 501 Required byte offset parameter bad or missing.

Then the upload begins again.

Any ideas?

Quote
admin
This is still related to the connection timeout. As the SmartFTP client you are using does not support the ALLO command for filesize it seems this causing the connection to time out for the larger file sizes. 

Can you extend the connection timeout from the default 30 seconds to say 1 hour (probably excessive but gives some room).

I believe the link below describes how to do that for your FTP client:


I believe what should then happen is that you will see connection to continue to remain upon until the file is actually fully complete.
Quote
goatlady
Thanks, changing the timeout fixed the issue
Quote

Add a Website Forum to your website.